Defending a network against the broad variety of modern threats is no easy task, and it can be made even more difficult if the security team doesn’t have a good picture of how attackers come at their infrastructure. That’s usually where penetration testers or red teams come in, but Uber has developed an in-house tool that provides a third option: automated adversarial simulation.
Known as Metta, the tool includes elements of work done on several other projects inside Uber that implemented some of the same concepts. At its core, Metta is designed to help defenders develop a more accurate idea of how their detection rules and other defensive measures hold up under attack. The company has released Metta as an open-source project and the code is available on GitHub.
Having external or internal teams perform pen tests or periodic red-team exercises is a time-worn concept and these tests can provide valuable information for defenders. But those operations also can be limited in scope and are necessarily time-constrained.
“As an emerging concept, the industry has yet to settle on a definitive definition of adversarial simulation, but it involves simulating [components of] targeted attacks in order to test both an organization’s instrumentation stacks and their ability to respond to the attack via their incident response process,” Chris Gates, a senior security engineer at Uber and one of the creators of Metta, wrote in a post introducing it Tuesday.
“I’ve come to believe that adversarial simulation really is where people need to go."
Automating the testing of detection mechanisms inside a network and being able to incorporate the results quickly has become a major challenge for many security teams, especially in larger organizations. But it’s also a vital part of a mature defensive program, and Gates said Uber considered using some outside tools to do the job but eventually settled on the idea of building one in-house. Gates said the main idea was to build a tool that could discover when the detection and alerting pipeline wasn’t working correctly and help determine where and how the failure occurred.
“Pipelines are complicated. Your laptop doesn’t just immediately talk to Splunk. There are usually a few hops in between there,” Gates said in an interview.
“We were having trouble with things going up and down and we weren’t sure why or when. We had a requirement to figure it out. Now, when there’s a problem, we can go in and find out why.”
Uber has been running Metta internally for about a year, and although Uber has a mature security program, Gates said many types of organizations should be able to use Metta.
“As long as they have some host or network instrumentation they can get some value out of it. There’s a lot of value there even at a basic level,” Gates said.
“I’ve come to believe that adversarial simulation really is where people need to go once they have decently mature teams. If you can do this daily, weekly, or monthly, it ends up being so much more valuable than a red team. I hope other people can get value from it.”