Chris Eng, chief research officer at Veracode, recently joined Dennis Fisher on the Decipher podcast to talk about the company's new State of Software Security report and trends in enterprise security. This is an edited and condensed transcript of the conversation.
Dennis Fisher: When you get this back, was there anything that jumped out at you as a surprise one way or the other?
Chris Eng: Well one of them was just the composition of applications and how that's changing and we kind of start out by looking at multiple language applications versus single language applications and there's this precipitous drop in the past three years of the percentage of applications that are uploaded to us that incorporate multiple languages and so basically the closer we get to the current time the more we're getting smaller single language applications which makes you think a lot about microservices and just the way that development has evolved in the past few years. And so as we were looking at that stat we actually lined that up with the Google trends search for the word microservices and you actually see that they're not right on top of each other but it's pretty close. Um, and so it was interesting to see that represented across our user base.
Dennis Fisher: Earth.
Chris Eng: That's the thing that's really cool about the report that we do is that a lot of the reports are survey based and you're asking people, What do they do or what do they plan to do. And it's probably directionally accurate but it still doesn't tell you that you people want to make themselves sound a little bit better than they are whatever. Ah, but this is like we're taking all the scans that people are actually doing on the platform over the course of the lifetime of these applications and we can see there's no lying right? You can see what happened, and so that was a cool finding. I think the other one that also kind of lines up with this move towards devops that we've been talking about is like the increase in how often applications are getting scanned. It used to be a few times a year if you go back a decade to the earlier days of this report. And now like 90% of applications are scanned like three times a week or more, which is exactly what you expect when you have security tooling baked into the development process when you kick off a build. You also kick off your QA test and your security test. And stands to reason that you would start to see it more frequently. But the data shows that, so it's nice to see.
Dennis Fisher: Is that also a function of how often apps are being updated and integrating new functionality?
Chris Eng: Yeah, part of it is, but development kind of was in front of security on that like so development started moving away from yearly or quarterly releases when they started using agile scrum. But the idea being you have a releasable product at the end of every iteration and iteration might be two weeks and so you could build a feature and release it. You don't have to release it. But it's releasable. I remember way back in the day there was this presentation by I think somebody at Flickr, I don't know if it was a security person or development person, but the title of it was like a dozen deploys a day or like multiple deploys a day or something like that, at a time when this it was unthinkable. Nobody was doing that, people were releasing quarterly. Development has been way ahead of security on that front and and I think part of it was the security tooling had to catch up to the point where it was easy to integrate into the tools that developers were already using. There's been a little catching up to do and I think we're seeing the outcomes of that as people start to adopt these same practices for their security work.
"We really need to mature our ability to even take inventory of what we have so that we know where to patch next time this comes around so that you never waste a crisis."
Dennis Fisher: You're also looking at how good organizations are at reducing their security debt. Are they getting better at that?
Chris Eng: They are getting slightly better. It's hard to do without a chart to point to but there is a great chart in the report that shows you what we call a survival curve and what it shows you is the probability at any point in time that a flaw is going to be fixed, and so what it shows you is, well it'll take this long to fix 25% of the known issues that I have, it will take this long on average to fix 50%, and so we can break that out by different product lines so we can say for example that it takes people longer to fix vulnerabilities in open source libraries than it does to fix vulnerabilities in their own code. And we can say, oh to fix half of the known third party flaws that are reported takes an average of three hundred and ninety seven days to get to the halfway point. But the really crazy thing is that that number to get to the halfway point for open source library flaws was three years. It was three years to get there in 2017 and today it's one year so both of those are terrible right? Both of those are awful but it's getting better.
Dennis Fisher: Why do you think it. It's so long to get those third party and open source things fixed? Is it a function of you know the maintainer is not getting the patches out quickly enough or is it just other obstacles?
Chris Eng: I think it's rarely the maintainer. It's rarely if ever the maintainers. Especially if it's a widely used library, they're usually pretty quick about those things because they’re not that they're well-funded but there's usually people that are pretty dedicated to the project and they jump on it. I mean there were not that many people that maintained Log4j but they were on that right away even with all the subsequent iterations. There are not many people on OpenSSL, if you remember from Heartbleed. They responded very quickly. But it takes a long time because honestly those are the choices that engineering teams make, not just we're not going to patch this. But, we're not going to maintain the hygiene of the project overall and by that I mean right when they first download the library, when they are building it into their project, they download it off the internet and then they incorporate it into the build and that's it. It works and then they go there. Meanwhile in the background, vulnerabilities are being discovered in the library but their product still works. So there's no forcing function that says, well you need to update this library because there are these known vulnerabilities. So now they get a version out of date, two versions out of date, three versions out of date. And then suddenly they're five years out of date and so the work required to get current is just monumental at this point. It's a lot of work. You're not going to get all the way out of the hole. What do you do? What do you work on first? Things that are actively being exploited in the wild is a good place to start.
Dennis Fisher: The other part of it that strikes me is that unless it's something like Log4j that gets an insane amount of attention and publicity, that you might not even know. I mean if you have dozens or you know several dozen open source or third party libraries in your product. How do you keep track of every one of those update cycles in security vulnerabilities unless you're just on every mailing list?
Chris Eng: Right? And your security team can't constantly be asking people to work through the weekends right? So you have to really know to pick and choose those carefully. I mean this was definitely a good one to use because you could see that it was being widely exploited. You could see that number one, publicly available code, number two, actually being exploited, and number three, super easy to exploit. Log4j and other incidents like that are something to point to and say we're lucky to get out of this one. We really need to mature our ability to even take inventory of what we have so that we know where to patch next time this comes around so that you never waste a crisis.