On a recent Ed TALKS, I was joined by three security experts with experience scaling AppSec programs. One of them is Dustin Lehr, who spent 13 years as a professional software engineer/architect, and officially transitioned into security nearly five years ago. Dustin has built AppSec programs for companies like Staples and Fivetran. He also runs a great online MeetUp called Let's Talk Software Security - an open forum that can be found at www.meetup.com/lets-talk-software-security.
His Keys to Scaling AppSec: Building Relationships and Security Champions
Building Relationships (aka getting teams to "play nicely")
A big part of building relationships for Dustin is meeting dev teams where they live, i.e., speaking their language. For this, he employs deliberately tactical methods, such as referring to bugs (vs. vulnerabilities) and using phrases like breaking the build instead of application availability. It has to do with building a foundation of common knowledge. After all, the security team is asking the dev team to do something other than develop… so why not accommodate them by adapting to their language? Another tactic includes finding a key ally or two to help spread the word from inside the dev ranks.
Watch Now: Ed TALKS: Scaling AppSec – Getting Tools to Perform
Modern application design and the continued adoption of DevOps expand the scope of automated security testing and push tools to the limit. At the same time, complex platforms like IoT and Blockchain require more specialized tools and skills. Hear how product and appsec pros plan to scale software securely in 2022.
Once you've made these moves, you must deliver. Demonstrate value. Show that you're providing useful information to the developers, answering their questions, and becoming a resource for them. Do the research and find them a good answer that considers the business side - this is where the importance of understanding both the business context and the technical software development angles becomes paramount to gaining trust. What is the business trying to accomplish? What is that dev team trying to accomplish specifically? Being solution-focused and tailoring that solution to those two contextual pieces is key to Dustin's success.
I found this quote from David Sims of Blue Cross Blue Shield interesting. He said, "As an industry, we train in silos and as a result we deliver in silos." Dustin's take on this is that he agrees, but to solve it, he says, we need to think of training as a two-way street. Security folks like to sit around and talk about what's best for developers, but that's all just speculation and theories unless we speak to the developers in our organizations. Most people receive very little security training as a software engineer, and most methodologies don't include security as an aspect of software quality. Instead of the definition of "done" for a particular task, meaning when one has met the functional spec, what about including some sort of security check as well? Better yet, expand the functional spec/requirement to include a security use case. We need to get to the point where we're building security into our thought process as early as the specification and requirements phase. Now THAT's shifting left.
We had an interesting question from an attendee who asked, "who should be the people vetting the tools? Is it the developers, security engineers, architects, leadership?" The answer here is all about inclusion. It's evident that trying to force-feed a tool chosen by one team in isolation is a much more difficult task than conducting a collaborative tools assessment & procurement activity. For security professionals, make the dev feel like they are part of the process, i.e., ask them for requirements before tools are even evaluated. This way, there is alignment in terms of what to look for and can avoid a subjective or hostile takeover of the evaluation process by someone(s) who felt left out. You may be surprised to learn that diverse teams genuinely make better decisions… that goes for role, age, race, religion, sex, et al. Simply including both teams in the process is likely to yield better results.
Build Security Champions
Building upon an attendee's comment, "Tools can help address short-term issues, but the problem is with educating users, customers, and staff on long-term business goals and objectives. Tools come and go, but the knowledge innovation for security & sustainability is what ultimately will make the difference."
This is also a topic near and dear to Dustin, who has rolled out successful security champions programs. My team at Security Innovation worked with Dustin at Staples and Fivetran to craft and execute those programs. Though Gartner published this (2020) after the Staples program was in operations, what they recommend to build a good security champions program echoes what Dustin oversaw: training that uses a belt system (like martial arts) and has different types of blended learning elements, e.g., online or instructor-led training coupled with some hands-on capture the flag exercises that are practical, all tied together with some specific on-the-job tasks directly related to each person's role.
Learn to Scale AppSec with a Security Champions Program
As organizations move toward a CICD and DevOps environments, every stakeholder needs to be at least security-aware in some respect. Security teams aren’t scalable, so the best way to scale AppSec is to spread the load.
Let's also level-set on what a security champion program is. In my experience, it is recruiting volunteers across the organization to represent security as part of their job (importantly, in addition to their day job). These non-security people you identify to help represent security in their respective group(s) is an effective method for driving culture change. One of the reasons it's so effective is because it's different when somebody on the existing team starts talking about security … you're driving change from within the teams as opposed to some external third party.
As both a software security training and assessment provider, this is a topic I'm passionate about. We see mistakes software teams make in our security testing services and frequently find vulnerabilities automated tools miss. I'm not anti-tool; we use them in our testing as well. However, tools are limited in the way they are programmed (most commonly pattern-matching.) Knowledge is power, and power is confidence.
Read Part 2 of Our 3-Part Scaling AppSec Blog Series
In the same episode of Ed TALKS, Joe Basirico, Senior Director of Product Security, Highspot, also provided some of his best tips for scaling application security.
Read Now: Scaling AppSec with Quick Wins, The Right Tools, and Purple Teams
In terms of building and running a security champion program, it is about starting with those allies mentioned above. Be on the lookout for people who are already interested in security. Some of the pitfalls would be just focusing on content and not necessarily focusing on motivation. Providing things like a belt program to help recognize people's efforts (maybe even engage HR to tie this to your professional development and job review process). The more effort they put into it, in a gamification fashion, they're essentially leveling up, which shows that they're putting in the work, making it easy to recognize people for their effort. Relevant content, structured to acknowledge progress, working from within each organization to incorporate security as part of their daily thought process
About Ed Adams, CEO
Ed Adams is a software quality and security expert with over 20 years of experience in the field. He served as a member of the Security Innovation Board of Directors since 2002 and as CEO since 2003. Ed has held senior management positions at Rational Software, Lionbridge, Ipswitch, and MathSoft. He was also an engineer for the US Army and Foster-Miller earlier in his career.
Ed is a Ponemon Institute Research Fellow, Privacy by Design Ambassador by the Information & Privacy Commissioner of Canada, Forbes Technology Council Member, and recipient of multiple SC Magazine’s Reboot Leadership Awards. He sits on the board of Cyversity, a non-profit committed to advancing minorities in the field of cyber security, and is a BoSTEM Advisory Committee member.
