Most doom-and-gloom and sky-is-falling scenarios in information security tend to be exaggerated, but the fact that a significant number of Internet of Things will stop working over the next year doesn't seem like one of those hysterical warnings.
IoT and smart devices will start break down and stop working because they will be unable to connect online, said security researcher and consultant Scott Helme. These devices won't fail because of simultaneous hardware issues or a coordinated widespread attack. It's a lot more simpler than that: the root certificates these devices use to establish secure online connections will expire.
The security of Internet connections depend on a web of trust between certificate authorities and digital certificates. There may be multiple levels of certificate authorities and certificates, each one validating the one below it. But at the most base level, all CAs eventually link back to (and derive their ability to issue certificates from) a Root CA. A client uses root certificate "quite literally embedded in your operating system or your browser of choice," to validate the server's certificate before establishing a secure connection with the server. "It's [the root certificate] physically present on your device," Helme said.
When the client can no longer connect to the server because its certificate has expired, the server administrator has to renew and update the certificate. This isn't unusual, as server certificates don't have long lifespans, and starting September, will be restricted to a validity period of one year or less. Root certificates were designed to have longer expiration windows--such as 20 to 25 years--because they are in every single client that connects to the Internet. Helme is concerned there isn't an equivalent fix in the reverse scenario, when the client cannot connect to the server because its root certificate has expired.
Time's Up
The poor state of client-side updates is a massive problem--many Android phones are not supported long enough to receive software updates and networked devices such as printers and routers don't always have a straightforward way to update the firmware. Many of the IoT devices on the market don't even have an update mechanism of any kind. When the root certificates on these devices expire, as they inevitably will because all certificates have some kind of expiration date, the devices will just stop working. The expiration date for many of the original root certificates is right about now, Helme warned.
"We're coming to a point in time now where there are lots of CA Root Certificates expiring in the next few years simply because it's been 20+ years since the encrypted Web really started up and that's the lifetime of a Root CA certificate," Helme said.
CAs have created new root certificates and distributed them in operating system and browser updates over the years. If the client has one of the newer root certificates--either because the manufacturers shipped the device with a newer one or the user has updated the device--then the problem isn't immediate. But if the device still has one of the original certificates, time's up.
First Set of Failures
Something along these lines have already happened, Helme said. Customers were surprised and confused when select Roku streaming channels stopped working on May 30th. Roku informed those customers of a "global technical certificate expiration" and directed them to manually install a software update. This was an unexpected situation, but relatively painless, because Roku has a menu option for users to find and download software updates.
"That exact time [30 May at 10:48:38 GMT] was when the AddTrust External CA [Certificate Authority] Root expired and brought with it the first signs of trouble that I've been expecting for some time," Helme said.
For many devices that don't have an easily accessible option in the Settings menu, this situation would have been far more challenging to resolve. The customer may have to download the update separately and figure out how to copy it onto the device, as used to be the case for older wireless networking equipment. Or what's more likely, it may be impossible to update the device at all. In that case, the customer's only choices are to ask the manufacturer to replace the device, buy a new device, or just stick with the dysfunctional unit.
The problem wasn't limited to just Roku devices, but to any device with the old AddTrust Root CA Certificate installed. Payment platforms Stripe and Spreedly also experienced disruptions on the same day because some root certificates had expired. The expired certificates were causing some API/web clients to fail, "notably OpenSSL and curl," Spreedly said in its incident response report.
"There is a whole load of stuff that broke because of this Root CA expiring," Helme said.
Google software engineer Ryan Sleevi, who has been heavily involved in standards work for certificate authorities and the future of digital certificates.
Newest Is Still Old
The BBC ran up against this problem recently from the server side. The media company had a new security certificate for its streaming services which was linked to a root certificate with a validity period from November 2012 to January 2038. However, a significant number of smart TVs on the market had the older root certificates, which meant those TVs would not be able to validate this certificate and connect to the streaming service. The fact that an eight-year old root certificate is still not on enough devices highlights the severity of the problem. The BBC chained multiple intermediate certificates as a temporary workaround to use the older root certificate until 2028, at which point, the hope is that enough Smart TVs would have updated to have the newer root certificate.
Smart TV manufacturers may be releasing updates, but if it takes ten years or so to address the problem of old root certificates, then it is not addressing the issue effectively. Android devices also have this issue, since vendors update (if they offer any updates) a select number of models, and only for a very short period of time.
"There is a significant portion of devices that are either lagging seriously behind on updates or simply aren't being updated," says Helme.
On a larger scale, legacy operating systems still in use in ATMs, airport terminal departure boards, and point of sale systems will be unable to connect, leading to "failures to provide services to customers," said Fausto Oliviera, principal security architect at Acceptto, a continuous behavioral authentication provider. Enterprises who have not "developed ways to ensure that they are able to update their devices will have to quickly come up with ways to prevent failure of service," Oliviera said.
From the consumer standpoint, this is a challenge, because it doesn't even matter if the device is top of the line and brand new if it is relying on an old root certificate.
"Just because a device was built in 2018, it doesn't mean the software wasn't already 6+ years out of date," Helme said.
Updates Needed
One potential date for another way of failures is Sept. 30, 2021, when the DST Root CA X3 certificate used by many Let's Encrypt certificates expires. Let's Encrypt has been trying to transition to different Root CA, but had to delay the process because many devices don't know the Root CA it is trying to move to. The ISRG Root was issued in June 2015 and became a trusted CA in August 2018. Devices that received an operating system or software update would know about ISRG Root CA, but if a device has not been updated since August 2018--such as an Android phone--then it wouldn't know about the new root CA.
Updating a root certificate isn't like updating the browser or operating system every month. It isn't even like updating a server certificate every year (or six months). Making a requirement that devices have to check and download an update to get the latest root certificate every few years--even five years would help address this problem--shouldn't be so onerous.
Manufacturers and service providers have to think about updating the root store--if their devices or software depend on root certificates for secure communications, then they need to update so that they are using newer root certificates. Relying on users to update, or know how to update, isn't going to work. Developers have to think about how they will update the root store.
"Simply replacing the Root Store with the latest version might give a device years more useful life or prevent your service being negatively impacted when the next Root CA expiry comes around," Helme said.