Security Awareness Training

    Using NIST's Phish Scale to Optimize Employee Training
    Using NIST's Phish Scale to Optimize Employee Training
     

    Security teams need to be resourceful to get a handle on email phishing attacks. NIST’s Phish Scale is a resource that can help train employees to avoid these scams.
    Security teams need to be resourceful to get a handle on email phishing attacks. NIST’s Phish Scale is a resource that can help train employees to avoid these scams.
     

    by Dr. Matthew Canham
    getty-working-from-train.jpg

    Key Points

    • Phishing email is increasing in volume and impact, even though more employees are being trained not to fall for these scams.
    • NIST began in late 2020 to offer a free tool to bolster employee training.
    • NIST’s Phish Scale enables security teams to tailor the degree of difficulty of training campaigns individually and companywide.
    • Phishing email is increasing in volume and impact, even though more employees are being trained not to fall for these scams.
    • NIST began in late 2020 to offer a free tool to bolster employee training.
    • NIST’s Phish Scale enables security teams to tailor the degree of difficulty of training campaigns individually and companywide.

     

    Running effective phishing training campaigns can be challenging. On one hand you want training to be effective and realistic, but on the other hand you don’t want to overwhelm users. Fortunately, the National Institute of Standards and Technology (NIST) has developed the Phish Scale, a free, effective and easy-to-use tool to help you rate the difficulty of simulated phishing campaigns.[i] Using this tool as a guide could not only help you optimize training but also provide specific insights into user strengths and weaknesses so you can focus on the most significant performance metrics.

    A Need for Better Metrics

    In a Mimecast survey, 55% of security professionals said their company had received more phishing email than ever last year. The malicious links and attachments in these emails can lead to ransomware, theft of data and other crimes. Yet security departments are constantly under pressure to justify budgets to defend against them.

    Senior managers want to know that budgeted funds are producing good returns on investment. This demand often manifests as pressure to demonstrate reduced click rates on simulated phishing campaigns. But while click rates are commonly used as a key performance indicator (KPI), there are other metrics to consider.

    As a case in point, a sure way to reduce click rates is to repeatedly send out the same simulated phishing email. Of course, this is not effective training because the reduction in click rates simply demonstrates that users have gotten better at identifying and avoiding a familiar phishing message.

    The NIST Phish Scale provides multiple KPIs that security teams can show to senior management as evidence of training gains. The scale uses two dimensions to determine how difficult it might be for an employee to identify a simulated message:

    • Observable Cues: How many typos, misnomers or other false notes in the simulated email might tip off an employee that it’s fake?
    • Alignment: How well is the scam message aligned to employees’ context, to fool them by mimicking common workplace processes or current company events?

    The values for these two dimensions are combined to create an overarching level of email difficulty.[ii] To help implement this scale, NIST has provided several resources described in a guide titled “A Phish Scale: Rating Human Phishing Message Detection Difficulty.”[iii] 

    Applying the Phish Scale

    The Observable Cues dimension of the Phish Scale includes five categories:

    • Error.
    • Technical indicator.
    • Visual presentation indicator.
    • Language and content.
    • Common tactics.

    Examples in these categories include spelling and grammar irregularities, inconsistent email address, logo imitation, threatening language and a limited-time offer. 

    The Alignment dimension includes five message considerations:

    • Mimicking a workplace process.
    • Workplace relevance.
    • Timing vis à vis other situations or events (such as a news story).
    • Engendering concern for NOT clicking.
    • Having been the subject of targeted training or specific warnings.

    Each of these considerations is rated as “extreme” (8 points), “significant” (6 points), “moderate” (4 points), “low” (2 points) or “not applicable” (0 points).

    To assess the difficulty rating, first count each instance of an Observable Cue (every individual instance of a spelling or grammar error for instance). Once you have the total, determine whether you have “few cues” (1 to 8), “some cues” (9 to 14) or “many cues” (15 or more). Next assign a corresponding number of points for each of the Alignment considerations and obtain this total to determine which category of Alignment the message belongs to: “high” (18 or above), “medium” (11 to 17) or “low” (10 or fewer). Finally, determine the difficulty level of the phishing email using Table 1, below.  

     

     

     

     

    Alignment
    Context

    Table  1: NIST Phish Scale, Email Difficulty

     

     

    High

    Very Difficult

    Very Difficult

    Moderately Difficult

    Medium

    Very Difficult

    Moderately Difficult

    Moderately Difficult

    Low

    Moderately Difficult

    Moderately Difficult

    Least Difficult

      

    Few

    Some

    Many

     

    Observable Cues

     

    Working Through an Example

    Using the phish example in Figure 1 below, we can count ten Observable Cues and then assign the following points for Alignment: “mimics a workplace process” (8 points), “workplace relevance” (8 points), “aligns with situations or events” (4 points), “concern for NOT clicking” (2 points) and “subject of targeted training” (0 Points), for a total of 22 points on Alignment.

    We now have a rating of “some” Observable Cues and a “high” Alignment. Referring back to Table 1 above, we see that this campaign would receive a “very difficult” rating.

     

    NIST Phish Scale_test.png
    Source: Canham et al[i]
    [i]Phish Derby: Shoring the Human Shield Through Gamified Phishing Attacks,” Frontiers

     

    Reporting to Senior Management

    Having two metrics to consider, difficulty and failure rate, means that you can demonstrate value through employee improvement, even if click rates remain stable.

    Documenting employees’ improvement in spotting difficult emails alongside sustained click rates is still evidence for improved performance, because users are performing equally well on more difficult emails. If click rates decline over time, you also have evidence that this represents a meaningful improvement in performance. 

    Better Training Results

    Using the NIST Phish Scale can give you the additional bonus of fine-tuning your phishing campaigns by considering the number of Observable Cues separate from the Alignment context. Having a close hold on the difficulty of emails allows you to tailor campaigns according to ability. Increasing the difficulty of campaigns to add challenge for your cyber resilience stewards, or decreasing difficulty for your repeat clickers, creates a more productive training experience for all users regardless of ability levels. 

    Monitoring the Alignment dimension of the Phish Scale can also provide you with insight into which job roles or departments are most susceptible to various phishing pretexts and allow you to adjust training for these users accordingly.

    The NIST Phish Scale was introduced in late 2020 as a free tool. You can also buy email security, training, reporting and testing platforms, tools and services with varying degrees of sophistication and integration.[1] For example, Mimecast’s Safe Phish tool lets security teams turn real-life, “de-weaponized” phishing attacks on your company into training exercises.

    The Bottom Line

    Email phishing presents a persistent risk to companies, whose employees are often targeted with messages containing malicious links, infected attachments and other exploits that can lead to ransomware, data theft and other crimes. NIST’s Phish Scale can help companies train their employees to avoid these traps.

     


     

    [1]Security Awareness Computer-Based Training Reviews and Ratings,” Gartner
     

    [i]The Phish Scale: NIST-Developed Method Helps IT Staff See Why Users Click on Fraudulent Emails,” National Institute of Standards and Technology

    [ii] “Categorizing Human Phishing Difficulty: A Phish Scale,” Journal of Cybersecurity

    [iii]A Phish Scale: Rating Human Phishing Message Detection Difficulty,” National Institute of Standards and Technology

    Running effective phishing training campaigns can be challenging. On one hand you want training to be effective and realistic, but on the other hand you don’t want to overwhelm users. Fortunately, the National Institute of Standards and Technology (NIST) has developed the Phish Scale, a free, effective and easy-to-use tool to help you rate the difficulty of simulated phishing campaigns.[i] Using this tool as a guide could not only help you optimize training but also provide specific insights into user strengths and weaknesses so you can focus on the most significant performance metrics.

    A Need for Better Metrics

    In a Mimecast survey, 55% of security professionals said their company had received more phishing email than ever last year. The malicious links and attachments in these emails can lead to ransomware, theft of data and other crimes. Yet security departments are constantly under pressure to justify budgets to defend against them.

    Senior managers want to know that budgeted funds are producing good returns on investment. This demand often manifests as pressure to demonstrate reduced click rates on simulated phishing campaigns. But while click rates are commonly used as a key performance indicator (KPI), there are other metrics to consider.

    As a case in point, a sure way to reduce click rates is to repeatedly send out the same simulated phishing email. Of course, this is not effective training because the reduction in click rates simply demonstrates that users have gotten better at identifying and avoiding a familiar phishing message.

    The NIST Phish Scale provides multiple KPIs that security teams can show to senior management as evidence of training gains. The scale uses two dimensions to determine how difficult it might be for an employee to identify a simulated message:

    • Observable Cues: How many typos, misnomers or other false notes in the simulated email might tip off an employee that it’s fake?
    • Alignment: How well is the scam message aligned to employees’ context, to fool them by mimicking common workplace processes or current company events?

    The values for these two dimensions are combined to create an overarching level of email difficulty.[ii] To help implement this scale, NIST has provided several resources described in a guide titled “A Phish Scale: Rating Human Phishing Message Detection Difficulty.”[iii] 

    Applying the Phish Scale

    The Observable Cues dimension of the Phish Scale includes five categories:

    • Error.
    • Technical indicator.
    • Visual presentation indicator.
    • Language and content.
    • Common tactics.

    Examples in these categories include spelling and grammar irregularities, inconsistent email address, logo imitation, threatening language and a limited-time offer. 

    The Alignment dimension includes five message considerations:

    • Mimicking a workplace process.
    • Workplace relevance.
    • Timing vis à vis other situations or events (such as a news story).
    • Engendering concern for NOT clicking.
    • Having been the subject of targeted training or specific warnings.

    Each of these considerations is rated as “extreme” (8 points), “significant” (6 points), “moderate” (4 points), “low” (2 points) or “not applicable” (0 points).

    To assess the difficulty rating, first count each instance of an Observable Cue (every individual instance of a spelling or grammar error for instance). Once you have the total, determine whether you have “few cues” (1 to 8), “some cues” (9 to 14) or “many cues” (15 or more). Next assign a corresponding number of points for each of the Alignment considerations and obtain this total to determine which category of Alignment the message belongs to: “high” (18 or above), “medium” (11 to 17) or “low” (10 or fewer). Finally, determine the difficulty level of the phishing email using Table 1, below.  

     

     

     

     

    Alignment
    Context

    Table  1: NIST Phish Scale, Email Difficulty

     

     

    High

    Very Difficult

    Very Difficult

    Moderately Difficult

    Medium

    Very Difficult

    Moderately Difficult

    Moderately Difficult

    Low

    Moderately Difficult

    Moderately Difficult

    Least Difficult

      

    Few

    Some

    Many

     

    Observable Cues

     

    Working Through an Example

    Using the phish example in Figure 1 below, we can count ten Observable Cues and then assign the following points for Alignment: “mimics a workplace process” (8 points), “workplace relevance” (8 points), “aligns with situations or events” (4 points), “concern for NOT clicking” (2 points) and “subject of targeted training” (0 Points), for a total of 22 points on Alignment.

    We now have a rating of “some” Observable Cues and a “high” Alignment. Referring back to Table 1 above, we see that this campaign would receive a “very difficult” rating.

     

    NIST Phish Scale_test.png
    Source: Canham et al[i]
    [i]Phish Derby: Shoring the Human Shield Through Gamified Phishing Attacks,” Frontiers

     

    Reporting to Senior Management

    Having two metrics to consider, difficulty and failure rate, means that you can demonstrate value through employee improvement, even if click rates remain stable.

    Documenting employees’ improvement in spotting difficult emails alongside sustained click rates is still evidence for improved performance, because users are performing equally well on more difficult emails. If click rates decline over time, you also have evidence that this represents a meaningful improvement in performance. 

    Better Training Results

    Using the NIST Phish Scale can give you the additional bonus of fine-tuning your phishing campaigns by considering the number of Observable Cues separate from the Alignment context. Having a close hold on the difficulty of emails allows you to tailor campaigns according to ability. Increasing the difficulty of campaigns to add challenge for your cyber resilience stewards, or decreasing difficulty for your repeat clickers, creates a more productive training experience for all users regardless of ability levels. 

    Monitoring the Alignment dimension of the Phish Scale can also provide you with insight into which job roles or departments are most susceptible to various phishing pretexts and allow you to adjust training for these users accordingly.

    The NIST Phish Scale was introduced in late 2020 as a free tool. You can also buy email security, training, reporting and testing platforms, tools and services with varying degrees of sophistication and integration.[1] For example, Mimecast’s Safe Phish tool lets security teams turn real-life, “de-weaponized” phishing attacks on your company into training exercises.

    The Bottom Line

    Email phishing presents a persistent risk to companies, whose employees are often targeted with messages containing malicious links, infected attachments and other exploits that can lead to ransomware, data theft and other crimes. NIST’s Phish Scale can help companies train their employees to avoid these traps.

     


     

    [1]Security Awareness Computer-Based Training Reviews and Ratings,” Gartner
     

    [i]The Phish Scale: NIST-Developed Method Helps IT Staff See Why Users Click on Fraudulent Emails,” National Institute of Standards and Technology

    [ii] “Categorizing Human Phishing Difficulty: A Phish Scale,” Journal of Cybersecurity

    [iii]A Phish Scale: Rating Human Phishing Message Detection Difficulty,” National Institute of Standards and Technology

     

    Subscribe to Cyber Resilience Insights for more articles like these

    Get all the latest news and cybersecurity industry analysis delivered right to your inbox

    Sign up successful

    Thank you for signing up to receive updates from our blog

    We will be in touch!

    Back to Top