How We AppSec @ Duo
Introduction
If there’s one thing I’ve noticed in my time working in the world of application security, it’s that each company has their own process and what they think is the “right” way to do things. Bringing security to your applications and hardening them against attacks can definitely be difficult to accomplish. Some companies have one or two people on the general security team that are part-time AppSec, others have full teams dedicated to the effort. There’s also some companies, usually smaller in size, that operate without any kind of Application Security team at all.
Here at Duo, we’re fortunate enough to have a team focused on the security of our wide range of applications (the web-based administration interface, mobile applications and SSO proxies – just to name a few). Our team spends our time working to improve the code, communication and processes around securing our offerings. One of our main goals is to provide this not as an independent operation, but as part of the development process. We spend time encouraging development teams to build security in and teaching them about writing secure code rather than just tossing issues back over the wall.
I’m getting ahead of myself, though. I wanted to shed some light on how our Application Security team works and some of the lessons we’ve learned along the way towards creating a more mature Application Security team.
Security Activities
One of the tasks that we, as the Application Security team, work on during any given day is what we call a “security activity.” Basically, this refers to us taking time to review a question or need from the development groups related to the security of our various applications.
Intake
These requests come in from several different channels and can vary widely in scope – all the way from a simple question in our public Application Security Slack channel to a more thorough review of a new feature or a major change in functionality for the application. This includes the reviews of code changes, evaluation of third-party tools, and ensuring the quality and safety of new packages the teams might want to add to our codebase.
These requests might also include general advice on how to proceed with a feature with the development team providing us with a plan of action before any development has been done. Having this push for application security up front is one thing that many application security and development teams struggle with, but it pays off huge rewards in the end.
Developer Coordination
Another key part of any activity is the coordination with the developers involved in the change or the request. Our Application Security program has a well-defined goal that we shouldn’t consider ourselves a separate organization from the development groups, but as a collaborator in their efforts. In fact, we’re even in the process of renaming our team from “Application Security” to “Security Engineering” to reinforce this.
We believe that a key factor in the success of an Application Security program is working collaboratively with the developers. We try to be consistently available to answer their questions and provide them timely feedback on their review requests. The size of our team helps make this possible, too. While we’re currently a small team, our ratio of developers to security engineers is relatively high. This means we can respond quicker and help the development groups stay on schedule with their sprint-based workflow.
Reporting and Follow-Up
Once we’ve completed the assessment, we write up a report of our findings complete with references to actual tickets filed for any issues we might have found.
By filing these tickets, we work in the same context as the development groups and decrease that friction even more. These reports also sometimes, when relevant, include a summarized listing of the ASVS items that were checked for the reference of the development team.
Once we’ve completed this report, we send it back over to the development teams and, if needed, schedule a follow up meeting to discuss the findings. We find this follow-up meeting important as it allows us to directly provide context and suggestions to the developers about the issues and clarify any additional questions they might have.
These meetings aren’t the end of the process either. As we strive for that engineering integration, we continue to help shepherd them, answering other questions that might come up during the development process. Sure, we want the applications we build to be as secure as possible; but we also want to make sure the engineers understand the “why” of the issues and any changes requested.
Developer Training
We also believe that education is one of the key factors in creating a strong culture of security among the engineers and the company at large. To help with this, we provide several internal trainings offered multiple times each year to help teach those interested about the concepts of application security and to get some hands-on work finding and fixing issues.
Right now, we offer two courses to help get engineers up to speed on the basics: an “Introduction to Application Security” course and an “Introduction to Threat Modeling” course. Both courses provide more general content shared via slides or other media and a large amount of time spent working on what they’ve learned in hands-on labs. These interactive labs allow them to apply the knowledge they’ve just learned, finding issues in a vulnerable application or creating a threat model for a simple tool.
In order to take things even further, we’ve also started work on an “Advanced Application Security” training to take the attendees beyond the basics shared in the introductory course and let them dig deeper into more advanced topics via an online learning platform. This platform provides them with lesson information and a live version of the code, walking them through the issue and showing them how to correct it.
We’ve found that this hands-on learning approach works much better than just bringing them in to a room and using a presentation-only approach. There always seems to be demand for the trainings in several of the offices too. We switch locations to try and make them as easy to attend as possible and there are efforts underway to try to make them more friendly to remote workers, making it even easier to attend.
Continuous Security
Finally, I wanted to share some about how we handle continuous security in our group. It’s one thing to work with the engineers up front and while the development is happening, but we want to ensure that the security level of the application remains constant and things aren’t introduced to heighten our risk levels.
The Application Security team can’t be in all places at once so we’re working on a security assessment pipeline tool. The goal behind this tool is to provide a way for both security engineers and those on the development teams to perform security-specific testing on their changes in a more automatic and standardized way. The ultimate goal is to provide them with the interface (a web UI) to be able to make it fully self-service to provide them with on-demand testing and tools.
This helps ensure the overall security levels of the code in the applications and also provides us with details about third-party modules with vulnerabilities and issues that already exist in the code. We’re building in tests for a wide range of technologies too including:
Python and Javascript syntax evaluation (static testing)
Automated dynamic testing of different Duo products
Static analysis issue de-duplication
And many more
Like any security testing program, our hope is that, as time goes on, more and more “green” will show on our dashboards and the “red” will consistently drop.
Another positive side effect of this testing and the results it provides is that the engineer running the test can see the issues reported and apply that knowledge to their work to avoid repeating the same mistakes.
Wrap Up
We don’t claim to have any kind of “secret sauce” for a mature, cross-team collaborative Security Engineering team, but we feel like we’re on the road for success. Our Director of Security Engineering, Mark Stanislav, has put together an excellent presentation about the work we’re doing here at Duo in the Application Security program and where it’s headed in the future.
I highly suggest checking those out. Who knows, maybe some of the thoughts in there will spark ideas in your own mind about improving (or creating!) an Application Security program in your organization.