4 Essential Phishing Metrics to Reduce Risk

A few phishing statistics you should know...

• 80-95% of all attacks begin with a phish (Comcast Business)
• Since November 2022 (the launching of ChatGPT) malicious emails have increased by 4,151% (SlashNext)
• The human element is involved in 68% of breaches (Verizon DBIR)
• With adaptive phishing training, employees globally improve threat reporting behavior by 9x in training and over 10x in real threat detection (Hoxhunt Phishing and Cyber Behavior Trends)
• 85% of security pros said that most employees know they are responsible for security but 59% of users either weren’t sure or claimed that they’re not responsible (Proofpoint State of the Phish Report)
• 92% of malware events are attributable to 3% of users (Cyentia Report: High Risk Users and Where to Find Them)
• The average cost of a phishing breach is $4.76M  (IBM / Ponemon Cost of a Data Breach Report)

Forget failure rate. What are the real 4 essential phishing metrics?

Anti-phishing training works when the security awareness manager ceases to be beholden to the phishing simulation failure rate, and starts measuring the key elements of good email security behavior: threat reporting volume, accuracy, and speed.

Most importantly, these metrics must be tracked in both the simulated and real environments.

Only then will we see whether the training is having real impact on human risk.

Cybersecurity experts agree that:

✅  The traditional security awareness training (SAT) model is ineffective...

✅  Anti phishing training is the key to human risk management.

This is an emerging category, so it’s critical to understand what the 4 essential metrics to shape and track an adaptive phishing training program:

• Simulated dwell time
• Simulated threat reporting
• Real dwell time
• Real threat detection

This article will explain why it’s critical to measure dwell time and threat reporting, and how connecting these metrics between training and the real world will measurably change security behavior, transform culture, and reduce the risk of a phishing breach by 10X.

‍

What makes anti-phishing training successful?

In a word, it’s phishing training that’s adaptive:

• Adaptive to the individual’s needs over time

• Adaptive to the ever-evolving threat landscape

• Adaptive to your organization’s goals

Adaptive phishing training transcends conventional models by automatically tailoring the learning experience to individual users’ backgrounds and skills even as they change over time.

We call them “adaptive” phishing training programs because they employ an AI/ML-enabled adaptive learning model: the curriculum automatically adjusts to users’ changing behavioral and learning profiles.

Put simply: the more skilled you become, the more advanced the training you'll receive

Adaptive cyber security training works best when its reward-based...

Participants might receive a quick micro-training and a digitally-induced dopamine rush for successfully completing the desired reporting behavior.

Think of it like a coach who motivates your cyber muscle development with a personalized training regimen.

Eventually, detecting phishing emails and threats becomes a habit.

These programs leverage AI to create hyper-realistic simulated phishing campaigns and automatically send the right simulations to the right people at the right time - sometimes dozens per year.

They sharpen detection and threat reporting skills and build a resilient security culture around a reward-based learning experience.

The 4 essential metrics of an adaptive phishing training program are:

• Phishing simulation reporting rate
• Phishing simulation dwell time (how long it takes to report a threat after it’s landed in
  your inbox)
• Real threat detection rate
• Real threat detection dwell time

As we’ll discuss below, phishing simulation failure rate is not an essential metric in an adaptive phishing training program.

‍

Evolving the security awareness training model

Traditional security awareness training (SAT) tools are designed, primarily, to be a quarterly tick-box compliance exercise...

And are usually based around failure rate.

The failure rate is simply the percentage of employees who fail to recognize or properly respond to simulated phishing attempts.

SAT tools generally offer a library of unchanging content that a security admin must manually pull from to create, distribute, and analyze a single campaign.

These solutions are highly manual and resource-intensive, which makes them challenging to scale.

SAT tools are also premised on phishing simulation failure rate.

They only deliver training once a user has clicked on a phishing simulation and is reprimanded in some way.

This failure-focused approach is a critical flaw in the SAT model.

Is failure rate important?

Yes, tracking failure rate is important... eventually.  

But phishing simulation failure rate is subordinate to threat reporting behavior and overall engagement rate.

Once threat reporting and engagement rates are high enough to provide a sizeable data sample, failure rate can show you your users' skill level at avoiding a phish.

A low failure rate doesn't necessarily mean your cyber security training is successful.

Your failure rate might depend on factors such as difficulty level, variety of the content, individual points of view, timing and frequency.

And so whilst you don't have to discard this metric entirely... it's worth remaining wary that it doesn't give you the full picture.

The SAT approach of restricting training to only those users who happen to open and fail a phishing email simulation provides scant training experiences to the organization by orders of magnitude.

It fails to address basic principles of behavioral science, e.g. rewarding good behavior.

A continuum of consistent practice locks in the desired behavior change far more effectively than periodically punishing people for failure.

Security habits are built on frequent training and positive reinforcement.  

Unlike the compliance-driven SAT model, adaptive phishing training is designed to change behavior and reduce human risk.

They do this by ingraining the key behavior of spotting and reporting phishing attacks.

‍

What is real and simulated dwell time?

Dwell time is the period between a threat infiltrating your network and its detection by an employee.

Essentially: how long a threat “dwells” unchecked.

In cybersecurity terms, it’s a race between attackers and employees, where every second counts.

Dwell time is not measured in a SAT platform, and only in a few adaptive phishing training platforms.

In our Phishing and Cybersecurity Behavior Trends Report 2024, we found that amongst 1.6 million users interacting with 15 million simulations and millions more real threats:

• Median dwell time in both training and the real thing improves after 1 year by over 32% with the introduction of an adaptive phishing training program

• The top 5% fastest employees report real attacks in less than 1 minute in training, and less than 2 minutes in real attacks.
• Dwell time is lower - meaning reporting is faster - in phishing training because simulated threats are sent during work hours.
• Very few platforms measure dwell time.

‍

Why is dwell time important?

Dwell time introduces the element of speed, and in cybersecurity, speed is essential.

The quicker a phishing attack is identified and reported, the less damage it can inflict.

Shortening dwell time directly contributes to containing and mitigating potential security breaches by alerting accelerating SOC mitigation.

Measuring and reducing dwell time is imperative.

Consider the “Fast 5” cohort: A SOC team can eliminate an entire phishing campaign within 2 minutes of its landing in emails with a timely threat report.

That means the threat will be removed from the system before hundreds or thousands of other employees can even open the malicious email.

‍

What is real and simulated threat detection rate?

Simulated threat detection is what happens when a user successfully reports a phishing simulation.

Within training programs, simulated threats help prepare users for the real thing.

The reporting rate for simulations gives you an idea of how people have engaged with your training.

These mock-ups mimic actual phishing attempts, enabling people with the skills and confidence to spot and report attacks.

At Hoxhunt, we advise our clients to aim for at least an average 70% reporting rate.

When over 70% of employees are reporting threats, the chances of them making an error and falling for an attack are significantly lower.

This reporting rate gives you a fairly solid indicator of the whole population’s level over time.

The reporting rate tells you that people are actively learning, and you can be more confident that they will do the right thing when they face real threats.

Real threat detection, on the other hand, is the application of this training in identifying genuine attempts to breach security.

An adaptive phishing program provides the tools to report real phishing attacks just as simply as in training.

Why is threat detection important?

Threat detection is the ideal outcome of a phishing attack.

Effective threat detection prevents the cyber-savvy users from clicking on phishing links, and saves their more vulnerable colleagues from even seeing those email threats in the first place.

Early identification and prevention of attacks

Threat detection gives security teams visibility into the threat landscape and enables prompt mitigation of dangers and incidents.

If you catch threats early, you'll be able to identifying vulnerabilities and potential attack vectors before they can be exploited by malicious actors.

There is no greater proof that a security awareness and phishing training program is working than an uptick in real reported threats.

These reports are not only removing threats from the system, but augmenting the quality of the Security Operations Center’s work.

Lower recovery costs

A swift response can be the difference between a minor incident and a catastrophic data breach (the global average cost of a data breach in 2023 was $4.45 million).

Costs might include things like recovery expenses, legal fees, regulatory fines and reputation damage.

Enhanced threat intelligence

Threat detection can give you can insight into attacker behaviors, techniques, and tactics.

If you're seeing certain types of threats more than others, this can be used to feed back into your training.

‍

How do you connect training to real-world impact?

You can connect training to real-world impact showing that what was learned in training is actually working when it counts most.

An impactful data visualization will show that as phishing simulation reporting rates go up, so too will real detected threats.

An adaptive training program is only as good as its measurable impact on actual threat detection and reaction times in the workplace.

‍

How much are cyber risks and costs reduced with an adaptive phishing training program built on speed and threat detection skill?

The 2023 IBM / Ponemon Cost of a Data Breach Study found a $1.2 million difference between breaches that were identified and contained before or after 200 days of initiation.

It also found that poorly trained vs. well-trained employees were the biggest cost-amplifiers and cost-mitigating factors: a $1.4 million difference per breach.

Speed and skill in cybersecurity behavior can save companies millions.

The biggest human cyber-risk is neglecting your humans...

95% of all data breaches are due to human error.

Through that fog of uncertainty, social engineers make billions by being better than the good guys at knowing what makes people tick and, accordingly, how to make them click.

Our research here at Hoxhunt tells us that 2/3 of active participants in adaptive phishing training reported a real suspicious email within a year, and they did so faster.

And we also found that there was also a 10X increase in both real and simulated threat detection rates:

• Simulated threat reporting jumps over 9X: from 7% in SAT programs at baseline to 66% after 1 year in the Hoxhunt adaptive phishing training program.
• Real threat detection jumps over 10X: from negligible at baseline to 33 threats reported per user, per month.

Cybersecurity is not really about the performance of technology - it's about the performance of people.

And like other industries that measure human performance, cybersecurity will be most improved by measuring human performance in terms of behavioral speed and skill.

For those in the frontline of IT security, this data isn’t just reassuring - it’s a call to action to adopt and refine human risk management techniques continually.

‍

Deliver personalized phishing training that people love with Hoxhunt

Here at Hoxhunt, we believe that people are best motivated to engage and learn when content is tailored to the individual.

Unlike your typical security awareness training solution, Hoxhunt automatically adapts the difficulty and content of phishing simulations to participants’ skills.

When someone is challenged to perform at the edges of their skills (not too hard, not too easy) they'll sharpen and expand those skills.

This is why organizations using our phishing training tend to see:

• 20x lower failure rates

• 90%+engagement rates

• 75%+detect rates

‍

🔑 Key takeaways: essential phishing metrics

• The research is clear: reward-driven, adaptive phishing training is the most effective means of changing behavior. This means training should be customized to individual users’ backgrounds and skills as they develop them over time.
• Employees who receive adaptive phishing training are 10X more likely to report real threats.
• Traditional security awareness training that relies on measuring failure rate doesn't work particularly well. Why? Because it only trains users who actually fail phishing simulations, instead of building positive habits with a reward system.
• Why do real and simulated dwell times matter? Dwell time is the period between a threat entering your network and it being detected. It's essentially a measure of how quickly employees can spot threats. Generally speaking, the quicker you catch breach, the less damage it'll cause.  
• Why do real and simulated threat detection rates matter? These phishing metric tell you what happens when a user reports either a simulated or real threat. An adaptive phishing program provides will allow you to report on real phishing attacks just as easily as simulated ones.

Want to dive deeper into phishing metrics? In the video below, David and Ryan from AES explain how a few changes skyrocketed reporting rates at AES and earned them an award at the annual CSO50 Conference 👇