CVE-2026-72897

Issue summary: A TLS server that calls SSL_set_SSL_CTX() to switch a connection to a different SSL_CTX part way through a handshake may access memory beyond the end of an internal array if the replacement context knows about more provider signature algorithms than the context the connection was created from. Applications which never call SSL_set_SSL_CTX() are not affected.

Impact summary: A remote peer may be able to cause a small out-of-bounds read, and in some circumstances a fixed-value out-of-bounds write, on the server heap. This may lead to a Denial of Service.

CWE: CWE-787: Out-of-bounds Write

Description: A TLS connection records how many certificate slots it has when it is created, taken from the SSL_CTX that created it: the built-in certificate types plus one slot for each provider TLS-SIGALG entry that context was aware of. That count sizes an internal array of per-slot certificate validity flags.

An application may replace a connections SSL_CTX part way through the handshake by calling SSL_set_SSL_CTX(), most commonly from a servername callback in order to serve a different virtual host. Doing so did not refresh the recorded count. A provider signature algorithms slot index is its position in the list of whichever context resolves it, so if the replacement context is aware of more of them than the original, an algorithm offered by the peer can resolve to an index beyond the end of the array. Processing the peers signature algorithms then reads one four byte word past the end for each such algorithm and, where the word read is zero, writes a fixed value over it. A peer offering many of them can corrupt heap metadata and abort the process.

Only provider signature algorithms which occupy one of the excess slots, and which the server also has configured, have this effect. Codepoints the replacement context does not recognise are discarded without being resolved to a slot, and provider signature algorithms are usable only from TLS 1.3.

The two contexts must therefore be aware of different numbers of provider signature algorithms, which requires separate library contexts, a provider loaded between the two being created, or providers which differ in what they advertise - in 4.0, for example, the default provider advertises SM2 where the FIPS provider does not. A deployment meeting the condition is also unable to negotiate the affected algorithms with legitimate clients, since the same stale count hides the corresponding certificates, so the misconfiguration is likely to be noticed. For that reason, and because the configuration is not the default, this issue has been assessed as Low severity.

FIPS impact: no No FIPS modules are affected by this issue as the affected code is outside the OpenSSL FIPS module boundary.

Weakness

The product writes data past the end, or before the beginning, of the intended buffer.

Affected Software

Potential Mitigations

• Use a language that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid.

• For example, many languages that perform their own memory management, such as Java and Perl, are not subject to buffer overflows. Other languages, such as Ada and C#, typically provide overflow protection, but the protection can be disabled by the programmer.

• Be wary that a language’s interface to native code may still be subject to overflows, even if the language itself is theoretically safe.

• Use a vetted library or framework that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid.

• Examples include the Safe C String Library (SafeStr) by Messier and Viega [REF-57], and the Strsafe.h library from Microsoft [REF-56]. These libraries provide safer versions of overflow-prone string-handling functions.

• Use automatic buffer overflow detection mechanisms that are offered by certain compilers or compiler extensions. Examples include: the Microsoft Visual Studio /GS flag, Fedora/Red Hat FORTIFY_SOURCE GCC flag, StackGuard, and ProPolice, which provide various mechanisms including canary-based detection and range/index checking.

• D3-SFCV (Stack Frame Canary Validation) from D3FEND [REF-1334] discusses canary-based detection in detail.

• Consider adhering to the following rules when allocating and managing an application’s memory:

• Run or compile the software using features or extensions that randomly arrange the positions of a program’s executable and libraries in memory. Because this makes the addresses unpredictable, it can prevent an attacker from reliably jumping to exploitable code.

• Examples include Address Space Layout Randomization (ASLR) [REF-58] [REF-60] and Position-Independent Executables (PIE) [REF-64]. Imported modules may be similarly realigned if their default memory addresses conflict with other modules, in a process known as “rebasing” (for Windows) and “prelinking” (for Linux) [REF-1332] using randomly generated addresses. ASLR for libraries cannot be used in conjunction with prelink since it would require relocating the libraries at run-time, defeating the whole purpose of prelinking.

• For more information on these techniques see D3-SAOR (Segment Address Offset Randomization) from D3FEND [REF-1335].

• Use a CPU and operating system that offers Data Execution Protection (using hardware NX or XD bits) or the equivalent techniques that simulate this feature in software, such as PaX [REF-60] [REF-61]. These techniques ensure that any instruction executed is exclusively at a memory address that is part of the code segment.

• For more information on these techniques see D3-PSEP (Process Segment Execution Prevention) from D3FEND [REF-1336].

References

• https://github.com/openssl/openssl/commit/00646e5085a0d12d29e0d2f9b9bc5f7111a50922
• https://github.com/openssl/openssl/commit/4135f553c9d3ba4a09fe752f5d30af2a6a092b2e
• https://github.com/openssl/openssl/commit/9c54d209486f6b1ad79fe2179c40f13200fa4f61
• https://github.com/openssl/openssl/commit/e87ed26b298a74d8ba61a53e9c7bcd1acac6b814
• https://openssl-library.org/news/secadv/20260929.txt