The OpenSSL Project has fixed a vulnerability in several versions of the software that could enable an attacker to cause a denial-of-service condition on target machines.
The bug is the result of an odd condition in the way that OpenSSL handles some elliptic curve parameters in certificates. In some cases, a function can go into an infinite loop when parsing those certificates.
“The BN_mod_sqrt() function, which computes a modular square root, contains a bug that can cause it to loop forever for non-prime moduli. Internally this function is used when parsing certificates that contain elliptic curve public keys in compressed form or explicit elliptic curve parameters with a base point encoded in compressed form,” the OpenSSL advisory says.
The vulnerability affects versions 1.0.2, 1.1.1, and 3.0, and is fixed in 1.1.1n and 3.0.2, which were released on Tuesday.
“It is possible to trigger the infinite loop by crafting a certificate that has invalid explicit curve parameters. Since certificate parsing happens prior to verification of the certificate signature, any process that parses an externally supplied certificate may thus be subject to a denial of service attack. The infinite loop can also be reached when parsing crafted private keys as they can contain explicit elliptic curve parameters,” the advisory says.
There are several different situations in which an OpenSSL implementation can be vulnerable to this bug, including TLS clients that are processing client or server certificates, hosting providers that are taking certificates or private keys from customers, or certificate authorities that are parsing certification requests from subscribers.
“In the OpenSSL 1.0.2 version the public key is not parsed during initial parsing of the certificate which makes it slightly harder to trigger the infinite loop. However any operation which requires the public key from the certificate will trigger the infinite loop. In particular the attacker can use a self-signed certificate to trigger the loop during verification of the certificate signature,” the OpenSSL advisory says.