The Biden administration’s new executive order, which mandates the development of standards that promote the “safety and security” of artificial intelligence systems, orders federal agencies to take a closer look at how AI could potentially impact areas like vulnerability discovery or the ways that it could make critical infrastructure systems more vulnerable to cyberattacks.
The level of popularity and hype surrounding generative AI - accelerated by tools like OpenAI’s ChatGPT, which was launched almost a year ago - has led to many technology companies racing to invest in this area. However, questions remain about the practical use cases for generative AI tools, what kind of profit they will generate, how they will potentially reshape various industries, and the long-term positive and negative impacts they will have. Cybersecurity professionals, for their part, have kept close tabs on the future capabilities of generative AI systems, and what they might mean both for defense teams and for threat actors.
The White House's executive order on Monday doesn't offer answers to any of these questions, but instead attempts to set the stage for developing and deploying what it calls “responsible AI,” directing the National Institute of Standards and Technology (NIST) to create standards, tools, and red-team testing guidelines for AI systems before public release. The hope here is to root out issues that might include "harmful or discriminatory outputs from an AI system, unforeseen or undesirable system behaviors, limitations, or potential risks associated with the misuse of the system." At the same time, the order mandates that certain AI system developers share their safety test results with the U.S. government.
“My Administration places the highest urgency on governing the development and use of AI safely and responsibly, and is therefore advancing a coordinated, Federal Government-wide approach to doing so,” according to the Biden administration's full text of the executive order released Monday. “The rapid speed at which AI capabilities are advancing compels the United States to lead in this moment for the sake of our security, economy, and society.”
“DHS will use the Cybersecurity and Infrastructure Security Agency's (CISA) cybersecurity best practices and vulnerability management process to increase the cybersecurity of AI systems."
As part of the executive order, the Department of Homeland Security (DHS) must create an advisory committee - called the AI Safety and Security Board - that looks at how the AI standards developed by NIST could be applied to the critical infrastructure sectors. The committee will also look at the potential risks that crop up from the use of AI in critical infrastructure sectors, like ways that AI could make critical infrastructure systems more vulnerable to critical failures or cyberattacks. At a broader level, the committee will discuss how AI could be used by the critical infrastructure community to improve security and incident response.
In order to support these efforts, within 90 days the head of each agency with “relevant regulatory authority over critical infrastructure” must report a list of potential risks related to the use of AI in those critical infrastructure sectors.
The executive order also outlines a vision to develop AI tools that would find and fix flaws in critical software, which builds on the White House’s AI Cyber Challenge, a two-year competition announced by the Biden administration in August that challenges competitors in the U.S. to identify and fix flaws using AI. Here, the DHS said it plans to partner with the Department of Defense in order to create a pilot program aimed at developing an AI capability for fixing vulnerabilities in critical U.S. government networks.
“The pilot program will also develop advanced monitoring of Infrastructure as a Service providers that use AI in critical infrastructure,” according to the DHS. “DHS will use the Cybersecurity and Infrastructure Security Agency's (CISA) cybersecurity best practices and vulnerability management process to increase the cybersecurity of AI systems.”
The DHS and Secretary of Defense have 270 days to report on the results of this pilot programs, including “a description of any vulnerabilities found and fixed through the development and deployment of AI capabilities and any lessons learned on how to identify, develop, test, evaluate, and deploy AI capabilities effectively for cyber defense.”
“Having coding assistants that reliably make more secure coding recommendations or even a more advanced ability to use generative AI to find and fix vulnerabilities would be a huge win for security."
Outside of cybersecurity, the executive order aims to pave the way for responsibly developing and deploying other future AI capabilities, touching on areas like fraudulent or deceptive AI-generated content and privacy rights.
The White House on Monday also announced that it has secured voluntary commitments from 15 companies involved with AI, including OpenAI, Microsoft and Meta. These companies agreed to test their systems ahead of release and invest “in cybersecurity and insider threat safeguards to protect proprietary and unreleased model weights.” Part of this commitment also includes giving attention to cyber capabilities for AI, such as ways potential defensive applications or ways that systems can help with vulnerability discovery, exploitation or operational use.
Nathan Hamiel, senior director of research at Kudelski Security, and Black Hat’s AI, ML, and Data Science track lead, said that the executive order “is a great start, and the risks identified are certainly ones to focus on.”
“Unfortunately, this is an executive order, so organizations aren’t compelled to comply,” said Hamiel. “Regardless, even though there’s no requirement to comply, focusing on the right foundational things, such as having NIST developing standards and tools and driving the development of tools that find and fix vulnerabilities in software, will definitely have a positive effect by their creation alone. Many organizations benefit from the NIST Cybersecurity Framework today and are under no obligation to use it, so it’s reasonable to assume the same here.”
Hamiel, who has focused on the security of emerging technologies like AI, said that right now there’s more hype around AI than there are practical use cases. However, generative AI still has some promise, said Hamiel, especially if it could be applied to the security of code or for bug hunting.
“Having coding assistants that reliably make more secure coding recommendations or even a more advanced ability to use generative AI to find and fix vulnerabilities would be a huge win for security,” said Hamiel. “That’s why it was nice to see this specifically called out in the executive order. These are still hard problems despite us already having tools that are capable of finding security issues in code, so I don’t expect this to happen immediately, but I’m hopeful. Even something more simple, such as using generative AI to generate fuzz targets for applications and help organizations work fuzzing into their development processes, is something people are doing today and many would find helpful.”
"It's good that people are thinking about this problem. I just wish the answer from the government wasn't red teaming. You can't test your way out of this problem."
Much of what the executive order is trying to accomplish are things that the software and security communities have been working on for decades, with limited success.
"We already tried this in security and it didn’t work. It feels like we already learned this lesson. It’s too late. The only way to understand these systems is to understand the data from which they’re built. We're behind the eight ball on this," said Gary McGraw, CEO of the Berryville Institute of Machine Learning, who has been studying software security for more than 25 years and is now focused on AI and machine learning security.
"The big data sets are already being walled off and new systems can't be trained on them. Google, Meta, Apple, those companies have them and they're not sharing. The worst future is that we have data feudalism."
Another challenge in the effort to build safer and less biased models is the quality of the data on which those systems are being trained. Inaccurate, biased, or incomplete data going in will lead to poor results coming out.
"We're building this recursive data pollution problem and we don't know how to address it. Anything trained on a huge pile of data is going to reflect the data that it ate," McGraw said. "These models are going out and grabbing all of these bad inputs that in a lot of cases were outputs from the models themselves."
"It's good that people are thinking about this problem. I just wish the answer from the government wasn't red teaming. You can't test your way out of this problem."
Dennis Fisher contributed to this story.