Hitting the Security Training Trifecta: Content, Metrics, and Governance

With the cybersecurity skills gap showing no sign of shrinking, organizations realize that what they are doing to reduce staff risk isn’t having the desired impact  - everything from peanut butter spreading of generic training to relying on old, monolithic methods. It’s time to rethink our approaches to skills development.

While organizations continue to chase the elusive Training ROI from a financial perspective, they should also consider MOI (Mitigation on Investment) from a security maturity improvement perspective. According to the Ponemom Institute Cybersecurity Training Benchmark study, training programs are shown to strengthen an organization’s security posture as measured by the Security Effectiveness Score (SES), a well-known indexing method unrelated to direct cost. The SES was established over a decade ago and is used as a yardstick in many reputable research reports, including the annual Ponemon Institute Cost of a Data Breach report. Organizations with higher scores indicate a more mature security posture.

This is not surprising – a ton of research data indicates similarly. Rather than rehashing the business case for security training, I want to focus on the need for blended (multi-modal) approaches and the key activities of high-performing organizations.

While many instinctually equate training with content, how organizations measure, deliver, and govern the training program are critical factors to success or failure. This is reflected in how the Ponemon Institute breaks down the 17 core activities into three core areas:

Organizations that have adopted 64% of the training practices are in the highest SES quartile.

Content – Make it Broad, Deep, and Relatable

While breadth and depth of coverage are compulsory for any training program, the learner’s ability to tie it into their day-to-day activities is essential. This is why creating a high-fidelity, relatable environment is critical in turning knowledge into on-the-job skills. Otherwise, learning feels like a hypothetical exercise.

“Learn by doing” enables skill mastery. Consider the unrealistic expectation that a child can learn to hit a baseball by simply just telling them about the technique. While it’s a great initial learning moment, they must practice “on the field” to turn that knowledge into skill. Simulation shows real-world consequences and can provide relevant feedback at critical learning moments. It encourages failure, experimentation, and trial & error. Equally unrealistic is putting pitchers and catchers through the same day-to-day training. While core baseball principles are the same for everyone on the team, each position has unique skills they need to master for their position, which is why realistic simulation and role-based training are the top two most impactful activities.

Realistic simulation and role-based training generate over 2x the ROI than other training elements on average.

Making the training relevant helps the learner feel it is meant for them. Attach training to actual events versus just referencing fictional or potential scenarios. This situational awareness creates a-ha moments where learners realize the impacts of their mistakes but don’t need to be. 

Your Development, IT, and InfoSec teams have many different roles. Training that cannot be related to one’s job or personal life (aka “nothing to do with me”) has lower engagement and retention. Job-specific scenarios enable learners to solve the real problems they are dealing with daily. It also helps minimize staff turnover. An older but still relevant study showed that 70% of participants confirmed: “job-related training and development opportunities influenced their decision to stay at their job.”

According to Learning Solutions, a leading industry publication for eLearning professionals: “Over and over, eLearning experts emphasize that in order to be engaging, training has to be relevant to the learners. Offering learners focused training that is directly applicable on the job, and, even better, creating personalized learning paths, will increase learner engagement. They will spend more time on the training, pay closer attention—and potentially improve their performance as a result.”

Lastly, ensure flexible training so staff can learn on their own time and pace. On-demand and self-pacing solutions can improve memory performance, particularly when the learner allocates more time to more complex material. It also serves as a handy refresher guide after new knowledge is attained (reusable).

Measurement – Feel Confident You’ve Reached New Heights

Measurement is the key to maturation and keeping on-the-bench time to a minimum. The path to success starts with KPIs and achievable goals – even if they are initially simplified and relative.

Baselining is essential – demonstrating mastery of specific skills (as opposed to passing a written/multiple-choice exam) and being able to compare to larger populations puts your program into a measurable state.

Simulated platforms like cyber ranges and other high-fidelity environments increase engagement and are more accurate assessments of real-world skills. By replacing assumptions with clear insights into strengths and weaknesses, you can focus on individual learning experiences and fill gaps with only necessary training.

Incentive-based programs are great ways to establish competency bars that employees can actually get excited about. Examples include belt and certification programs that increase in proficiency and are often tied to specific roles. These programs are win-win scenarios – learners are self-motivated (vs. organizationally mandated) to learn, and companies reduce risk via an informed staff.

Governance & Delivery – Get Everyone Vested

Knowing the program has C-level visibility connotes a level of importance, seriousness, and commitment to learners. This was a key success driver for our client Accenture, who noticed significant spikes in enrollment and consumption when management announced the program and sent encouraging communications.

Spikes in learner enrollment and consumption following program announcement and encouraging communications from management.

There is much debate regarding the efficacy of mandatory training. One school of thought is that you can’t force someone to absorb knowledge they don’t want to. However, mandatory training can also be motivational. Imagine the automotive industry saying that training is optional – the safety of life is then just an afterthought and not as inspirational. Making it mandatory but reinforcing the why is critical. Technical and security professionals are inherently curious and want to do the right thing – give them a reason to.

Training programs need to be ongoing and continually updated with the latest threat intelligence. Nothing tells employees you don’t take training seriously more than delivering outdated content once a year. 

Most important, and often overlooked, is ensuring skills are not lost and continuously honed. One of our clients had over 5K learners take on our ShadowBank cyber range, an intentionally vulnerable Web application designed to build an attack mentality and assess AppSec skills. With continued play beyond their initial training period, they yielded a 37% increase in score.

Lastly, while self-paced learning is important for scaling knowledge and flexibility, incorporating intimate live environments builds teamwork. This is why instructor-led (even if remote) and group awareness events are highly effective. This is also why options for train-the-trainer or apprenticeship programs are critical to organizational maturity and buy-in. This concept is often deployed in software development teams with the building of Security Champions, whose goal is to scale AppSec knowledge and reduce the dependency on the security team (and the friction that often goes along with it). Being trained by “one of your own” brings a fresh perspective and increased willingness to listen.

About Ed Adams, CEO
Ed Adams is a software quality and security expert with over 20 years of experience in the field. He served as a member of the Security Innovation Board of Directors since 2002 and as CEO since 2003. Ed has held senior management positions at Rational Software, Lionbridge, Ipswitch, and MathSoft. He was also an engineer for the US Army and Foster-Miller earlier in his career.

Ed is a Ponemon Institute Research Fellow, Privacy by Design Ambassador by the Information & Privacy Commissioner of Canada, Forbes Technology Council Member, and recipient of multiple SC Magazine’s Reboot Leadership Awards. He sits on the board of Cyversity, a non-profit committed to advancing minorities in the field of cyber security,  and is a BoSTEM Advisory Committee member.