A recent update to the widely used OpenSSL library introduced a new vulnerability that can be used for remote code execution on some specific x64 systems.
The bug is buried deep in the library’s code and is related to the way that OpenSSL does some calculations. It affects OpenSSL 3.0.4, which was released last week, but it’s only present on x64 systems with the AVX512 instruction set. Researcher Guido Vranken discovered and reported the original bug at the end of May, and the vulnerability was fixed in a release on June 21. That bug affected several separate code paths in the library, and the fix for one of those code paths did not have the intended effect.
“The fix that was applied to the dual 1024 RSAZ code is wrong because the reduction function is called with num set to the bit size, where it should be number of BN_ULONGelements (which are always 8 bytes large, because that is the size of an unsigned long on x64 systems, which is the only architecture which can have AVX512 support). So with the input sizes being 1024 bits, 8192 bytes are accessed (read from or written to) instead of 128,” Vranken said in a blog post Monday.
A fix for the newer vulnerability has been created and merged into the OpenSSL master but it has not been released yet. Vranken said the bug is trivially exploitable and the OpenSSL team is aware of the issue, but has not scheduled an updated release yet.
“OpenSSL makes heavy use of function pointers. Running find -name '.c' -exec grep 'METH. = {' {} ; | grep -v test from the repository root shows over 130 data structures that encapsulate a set of function pointers. Delta subtraction may be useful in exploiting this circumstance to make OpenSSL misbehave to varying degrees of severity,” Vranken said.
“Apart from code execution, there can also be scenario’s where private data is leaked to the attacker.”