Perhaps no one on the planet knows more about designing and implementing effective vulnerability disclosure and bug bounty programs than Katie Moussouris, and in nearly 15 years of doing that work, she’s seen those programs fail in every conceivable way. The failures come in many shapes and sizes, but the root cause is often the same: The organization went from zero to 100 with little or no preparation.
When Moussouris, a longtime hacker and the founder and CEO of Luta Security, finally succeeded in convincing Microsoft officials to start a modified version of a bug bounty when she was there in 2012, it was after years of agitation, education, and presentations about the value that it could bring to the company. Then, as now, Microsoft received more inbound bug reports than any other organization, and it didn’t make much sense for the company to start paying for them when they showed up by the hundreds of thousands for free.
“Why dangle cash in front of that firehose?” Moussouris said during a talk on bug bounties at the Black Hat USA conference Thursday.
No reason, the Microsoft team concluded. So instead, Moussouris and her colleagues came up with the idea of paying for offensive techniques. The BlueHat prize, as it was called, was a novel way of funding offensive research. The top prize was $200,000, which was a lot then and would still be a tremendous bounty payout today.
“So originally, I set the top prize for $200,000, which it was, and the Microsoft folks were like, "Why does it have to be $200,000? Why can't it be $100,000?” And I looked across the table and I said, "You know as well as I do that marketing spends more on that Black Hat-DEF CON party than $100,000,” Moussouris told Decipher last year.
“So if you're telling me that a night of drinking fun is worth more to you, Microsoft, than an entire platform level architectural mitigation, then we just definitely have to understand your priorities more." And they just kind of said, "Okay, 200k it is." I'm like, "That's what I thought."
The BlueHat prize was a success and led to Microsoft establishing a number of permanent bug bounty programs over the next few years, beginning with a $100,000 bounty in 2013 for offensive techniques that bypassed the latest exploit mitigations, along with a bonus for defensive techniques that could stop those mitigation bypasses. While most organizations don’t have the financial or human resources that Microsoft does, the value of laying a solid foundation and having a goal in mind before starting a bug bounty program is universal.
“We launched with specific strategic goals in mind,” Moussouris said.
Those goals did not include rushing a program into the world or jumping on a trend. The goals were to gain insights into novel offensive techniques and develop methods for protecting against those techniques. Too often, organizations look at a bug bounty program as a relatively inexpensive way to audit their code and gain some credibility in the security community and they start long before they’re ready. This is suboptimal and usually undermines the program from the beginning.
“Bounties can’t fix an organization’s lack of commitment or people. Security processes are specific to ingestion of bugs. You can get bug indigestion with a lot of these programs,” Moussouris said.
“Understanding your vulnerability intake process at this point in time is more important than how much you’ve spent or how many bugs they’ve found. Knowing is less than half the battle.”
The other portion of the battle is having processes in place to handle the intake, triage, and processing of the bug reports, and dealing with the researchers who report them. That requires knowledgeable, dedicated people who can handle the potential stress and strain of these programs, especially in large organizations that may see high volumes of reports. Without that infrastructure and a plan to support it, failure is not only an option, it’s a near certainty.
“Plan for workforce attrition. At eighteen to twenty-four months you will lose key people, and the program will collapse under its own weight,” Moussouris said.