Researchers at the University of South Florida have shown that long-standing approaches to training employees to recognise phishing scams may not be the most effective method.
Companies often send out simulated – or fake – phishing emails to employees to see who takes the bait and clicks. Those who fall for such scams typically receive an on-the-spot lesson designed to help them recognise suspicious messages in the future.
These phishing simulations — known as embedded training because once users fail, they are sent into training mode — are widely considered to be best practice in the cybersecurity profession.
However, new research co-led by the University of South Florida’s Muma College of Business faculty has found that this approach might not be the most effective way to help employees learn from their mistakes.
The findings, published in a paper co-authored by Dezhi Yin and Matthew Mullarkey of USF’s Muma College of Business, Gert-Jan de Vreede of Stevens Institute of Technology, and Moez Limayem, president and professor at the University of North Florida, identify two shortcomings associated with embedded training:
- Instant feedback can be limited in reach: only those who were duped received training, while those who passed may end up falling for a real phishing attack later, the research showed.
- Catching employees at the exact moment of failure – known as ‘just-in-time’ training – can be counterproductive: such on-the-spot training can lead to negative reactions in employees who feel exposed and may become defensive.
Instead, the researchers recommend taking a non-embedded approach. By providing feedback to everyone after the entire simulation ends, the exercise becomes a broader and more positive learning opportunity.
The study employed three large-scale experiments using a real phishing simulation platform. Thousands of students received realistic but simulated phishing emails. Some received immediate feedback after clicking, while others received follow-up messages days later. The team then tracked how likely participants were to fall for future simulated scams over the following weeks and months.
Among the study’s key insights, researchers discovered:
- Sharing lessons with the entire group, not just those who were duped, helped participants recognise scams more effectively and remain alert for months afterwards.
- Training does not need to be delivered at the point of failure to be effective: a time-delayed but more inclusive approach ultimately builds a stronger defence against real attacks.
