Yahoo studied employee responses to simulations to better understand how to make them take cybersecurity seriously. To make meaningful change, managers should take three key steps. First, they must identify critical employee behaviors. Second, managers must measure behaviors transparently. Finally, managers must use awareness to explain why something is important.
Telling your employees that they should do something isn’t enough to inspire meaningful change. Just ask any employee who has ever watched a cybersecurity awareness video. Although the videos instruct employees to be mindful of data security, they seldom lead to a wholesale improvement of a company’s security behaviors. To improve your cybersecurity culture, and, ultimately, your businesses’ resistance to attacks, you must measure what people do when no one is looking.
At the end of last year, the Cybersecurity at MIT Sloan research group (CAMS) began collaborating with Yahoo’s security organization, nicknamed the Paranoids, to understand how they’ve applied managerial mechanisms to influence the company’s cybersecurity culture. The Paranoids’ Proactive Engagement team have successfully employed several interesting and innovative mechanisms that led to better cybersecurity behaviors.
A Model of Proactive Engagement
In the summer of 2018, amid a reorganization of the larger security organization, the Paranoids brought together two disparate groups: the red team (a slick group of hackers that offensively tests internal systems, services, processes, and people to discover systemic weaknesses) and the company’s security awareness team. Later, the Paranoids added the behavioral engineering team, which focused on measuring the activities they’d deem as good security behaviors based on a mixture of HR data and enterprise technology logs.
To better understand how employees responded to cybersecurity threats, the behavioral engineering team’s first distinguished between employee actions, habits, and behaviors. An action, they concluded, was something a person does to completion. For instance, Yahoo employees were required to take an annual security training course. The desired result, taking the class, is an action. A habit was a shortcut made for repeatable actions. Training employees, for instance, to rely on a password manager rather than manual password changes can lead to a formed habit.
Finally, they defined behaviors as the combination of actions and habits within the context of a situation, environment, or stimulus. In the prior example, the desired security behavior is not simply to get employees to use a password manager. Instead, the goal was getting employees to generate and store credentials using a password manager whenever they were creating or updating accounts.
The Process of Changing Behavior
Attempting to change a behavior meant first identifying the specific context for a desired action. The Paranoids called this the creation of a behavioral goal. When creating a behavioral goal, the behavioral engineering team aimed to answer the question: “In which specific context do we want a specific cohort (or person) to do what specific action?”
For example: “When generating a new single sign-on password, we want all employees to generate and store the password within our corporate approved password manager.” The team’s ability to define these goals was key to effectively measuring the direction of cybersecurity culture within the organization.
As the behavioral engineering team studied and developed behavioral goals, a formula took shape.
Read more about
Step 1: Identify the desired behavioral goal. A clear goal for a specific behavioral outcome is a prerequisite for any measurable change to occur. The goal avoids what the team called “impossible advice,” which is any security guidance that requires the end-user to make a qualitative judgment about security.
Step 2: Find an appropriate measure and create a baseline. To improve a company’s cybersecurity culture, and enrich a businesses’ resistance to attack, one must measure what people do when no one is looking.
Step 3: Take actions to affect the measured behavior, adjust those actions over time, and repeat the process. Activities were then designed to impact the baselines. But equally important to the success of driving appropriate behaviors was learning from the results of these activities and then adjusting and creating new activities for continual improvement.
The process became the bedrock for the behavioral-change-based experiments the Proactive Engagement team conducted. Rather than instruct employees to determine if a link was suspicious, which is a subjective and flawed approach to cybersecurity, the Proactive Engagement group defined a new behavioral goal for employees: When your corporate account receives an email sending you to a website asking you to enter credentials, report the email to our defense team.
Measuring Employee Behaviors
Over and over, in red team operations employees would fall for phishing emails that presented them with fake login pages, just like the one that duped then-DNC chairman John Podesta’s assistant into typing his password into a fake login page obscured by a shortened link in a malicious email.
The team studied the problem and highlighted three key measures:
Susceptibility Rate: the number of employees who entered credentials and did not report phishing emails divided by the total number of phishing simulation emails sent.
Credential Capture Rate: the number of employees who entered credentials (and did not report the link to our defense team) divided by the number of employees who opened the phishing simulation and landed on the fake login page.
Reporting Rate: the number of employees who reported the phishing simulation divided by the number of total simulation emails sent.
With a behavioral goal and key measures defined, the team set out to implement new managerial mechanisms to diminish the rate at which employees gave up credentials. At the time, the phishing simulations were capturing nearly one out of every seven employees’ credentials at every test. One out of every 10 employees were accurately reporting the original simulation email as a potential phish. After looking at the data, the Proactive Engagement team decided to focus on stopping employees from entering their credentials on a phishing page.
The solution was already in place. They wanted employees to use the password manager that had already been paid for and provided by Verizon. Because the password manager will only auto-fill passwords on sites it recognizes, not the fake ones meant to steal credentials, it took the guesswork out of the hands of the employees.
Choice Architecture, Incentives, Communication, and Gamification
By the middle of 2019, the team installed the corporate password manager as a domain detection tool in its corporate-managed browsers and it made using the tool the default option for all employees. The team also offered incentives for active corporate password manager usage. Employees who actively used the password manager received merchandise such as Paranoid-branded t-shirts, hoodies, and hats. They also created how-to videos and content to educate users on what to look for, how to identify suspicious emails, and what to do if they saw something suspicious. These communications were paired with emails that nudged those who were duped by phishing simulations to read additional education materials and directed them to the corporate password manager.
The Proactive Engagement team measured progress by creating dashboards where managers could benchmark their corporate pillar’s performance against that of their peers. The dashboards were an important tool for managers because they created an environment of active and passive competition. The competition provided an incentive for employees to do better, and the dashboard allowed managers to see how their reports were doing. They also served as a bridge between the Proactive Engagement team and senior Yahoo leadership.
Actionable Recommendations for Managers
To make meaningful change, managers should take three key steps. First, they must identify critical employee behaviors. The biggest transformation the Paranoids undertook was organizational, not technological. They tested employees to better inform their strategy for changing cybersecurity culture. Only then did they develop and implement a plan.
Second, managers must measure behaviors transparently. While the security team couldn’t make business decisions, business leaders could. To get them to do that, the Proactive Engagement team built dashboards that allowed managers to benchmark their direct reports’ behaviors against that of their peers’ corporate pillars.
Technology & Innovation
Must-reads from our most recent articles on technology and innovation, delivered once a month.
Finally, managers must use awareness to explain why something is important. At no time did the Proactive Engagement team punish employees or mandate adoption of specific tools. Rather, they used their offensive testing capabilities to ground their advice in real-world attacks and then explained why those behaviors made sense for the business.
By the second half of 2020, the rate at which Yahoo employees’ credentials were captured in phishing simulations had been cut in half. The number of accurately reported phishing attempts had doubled. And most importantly, the usage of the company’s corporate password manager, the centerpiece of the company’s cybersecurity culture, had tripled.