One of the truisms of intelligence work is that the public never hears about the successes, only the failures. In some ways, security research is the inverse of that, with the successes celebrated publicly while the failures remain hidden.
Rarely do researchers provide a look at the times they run into a wall, go down the wrong path, or waste days of work because of flawed assumptions. But that’s just what Maddie Stone did on Wednesday, spending a large portion of her Black Hat session on reverse-engineering zero day exploits used in the wild detailing several instances in which she had failed, sometimes in painful fashion. This wasn’t a form of public confession or self-flagellation, but rather an illustration of the importance of perseverance and how much can be learned from failure.
Stone is a member of Google’s Project Zero research team, a group that specializes in finding zero day vulnerabilities in all kinds of software, whether it’s Google’s own apps, Windows, iOS, or something more obscure. A skilled reverse engineer, Stone was previously a member of the Android security team before hopping the fence to join Project Zero, where she focuses on root cause analysis of zero days that have been used in the wild. The goal of that kind of analysis is three-fold: finding the vulnerability, figuring out how to trigger it, and finding a way to exploit it. None of those is an easy task on its own, and succeeding at all three can be maddeningly difficult.
But if it was easy, everyone would be doing it. And everyone is not doing it.
Last December, Stone took on the analysis of an elevation of privilege vulnerability in Windows that Microsoft had just patched. The bug (CVE-2019-1458) had been exploited in the wild and researchers at Kaspersky had published an analysis of the flaw and some of the details of the exploit that had been used against it. But Stone did not have a copy of the full exploit to analyze, so she decided to rely on binary patch diffing to find the root cause of the vulnerability. In short, she needed to figure out what had changed between the older, vulnerable version of Windows and the patched version. And to complicate matters further, Stone did not have any previous experience with Windows.
“This was my first time ever looking at Windows. It was a gap I wanted to fill and this seemed like a good time to jump in,” Stone said.
And so she did. The vulnerability affected both Windows 10 and Windows 7 and on the advice of a colleague, Stone decided to focus on Windows 7. She set up a test environment and began installing all of the updates leading up to September 2019, which was the last available update before the patch for the vulnerability was released. Or so Stone thought. It turned out that there was another update in November, but Stone didn’t discover that until much later, after she had gone through the entire process of analyzing the vulnerability, getting a working proof-of-concept exploit, writing a lengthy blog post about it, and sending the post to Microsoft. That’s when Stone found out she had analyzed the wrong vulnerability.
“Microsoft said, 'Nice work, but this is the wrong bug’,” Stone said.
Instead of digging into CVE-2019-1458, Stone had been analyzing a separate bug, CVE-2019-1433.
“I thought that September 2019 was the latest because it was the only update shown to me, besides December 2019, when I clicked “Check for Updates” within the VM. Because I was new to Windows, I didn’t realize that not all updates may be listed in the Windows Update window or that updates could also be downloaded from the Microsoft Update Catalog,” Stone wrote in a blog post in April.
“When Microsoft told me that I had analyzed the wrong vulnerability, that’s when I realized my mistake. CVE-2019-1433, the vulnerability I analyzed, was patched in November 2019, not December 2019. If I had patch-diffed November to December, rather than September to December, I wouldn’t have gotten mixed up.”
Although she had made a key mistake, Stone said the experience of reverse engineering the vulnerability was still a valuable one and she learned things she might not have otherwise.
“I hope no one gets discouraged if you have failures early on. Just keep going and keep learning,” she said.
“There’s a lot of innovation that’s happening in this process that we can share with each other.”