What good is a phishing test if non-participation is considered success? Solely concentrating on lowering failure rates will give you an inaccurate picture of your business's risk profile. Want a truthful, honest look at your risk profile? Calculate your true risk!
True Risk vs. Measured Risk: The risky difference
Measuring true risk of a phishing attack breach is helpful. Knowing the actual likelihood of your people clicking something they shouldn’t—or reporting something they should--will guide good business and security decisions for the CISO and the C-suite. But “measured risk” of a phishing attack breach can actually be dangerous. The metric is typically based on phishing attack simulations’ pass/fail rates: did they click on the bad link, or didn’t they?
Risk measured solely by the click rate is a mirage. It can be based on a poorly executed internal campaign, or on ineffective training content. Or, sometimes, measured-risk-via-click-rate is a vanity metric designed to make vendors and security teams look good while lacking adequate sample size or context. Reporting risk to the board based on an empty metric is basically serving them junk food with empty calories; the sugar rush of saying, “Everything’s great!” will crash as soon as something bad actually happens and your team is held accountable for a suboptimal risk assessment.
What is the measured risk of a phishing attack breach?
Employee phishing simulation pass / fail rates calculated in a vacuum. If only 100 employees in a 1000-strong workforce are participating in training, then the sample size renders their results—positive or negative—inadequate. Also, remember that a phishing tool can be designed to show improvement. What does that mean?
Hard content that gets easier; or content that doesn’t effectively change, so the test takers can anticipate it and game the system. And the training itself is usually delivered via punishment-by-added-cybersecurity training, which discourages active participation. When the golden metric of an awareness tool is pass/fail rates come hell or high water, then the concept of that tool is fundamentally flawed.
What is the True Risk of a phishing attack breach?
Only when employee engagement of phishing awareness training is at a level of at least 50% of the organization, and ideally above 70% can the CISO calculate resilience with confidence, by dividing engagement rate by fail rate. A score of 14 is excellent and worth striving for, while above 10-12 still provides your organization competitive advantage. The Platonic ideal of 20-40 is rare, but possible.
Mind you, the engagement must be real. It can’t mean someone took one test, passed, and then was removed from testing but remains counted as a participant. Simulations must be challenging, and touch the upper echelons of the organization, just as sophisticated spear phishing and whaling attacks do. Engagement cannot be faked or taken for granted. People need to be constantly stressed with true-to-life threat simulations that evolve along with the threat landscape. Only then do pass / fail rates of threat simulations provide meaningful data for the infosec team to report to executive leadership with confidence.
