22.1 SMACK

In 2015, a group of French security researchers with Benjamin Beurdouche systematically tested the then-popular open source TLS implementations for state-machine-related bugs and uncovered multiple critical security vulnerabilities that have been dormant in these libraries for years [28]. They called these vulnerabilities State Machine Attacks on TLS (SMACK).

In several previous chapters, for example, in Chapter 6 Transport Layer Security at a Glance, Chapter 8 Elliptic Curves, Chapter 12 Secrets and Keys in TLS 1.3, and Chapter 18 TLS Cipher Suites, we learned that TLS supports a variety of protocol versions, authentication modes, key exchange alternatives, and protocol extensions.

TLS implementations typically consist of functions for parsing and generating messages, and for performing cryptographic operations. The message sequence – which message to expect or which message to generate as a response – is managed by the TLS peer’s state machine, which accepts or transmits the next message based on its state and the negotiated parameters for that TLS session.

The researchers observed that most TLS cipher suites and protocol extensions are defined in their own RFCs. As a result, the composite state machine that a TLS library has to implement is not standardized. This, in turn, requires developers implementing TLS to merge specifications from different RFCs into a single state machine that correctly switches between the protocol modes. This task is, however, error-prone, as illustrated by a simple example from [28] depicted in Figure 22.1.

Suppose a developer implements a (fictional) TLS cipher suite where the TLS client sends a Hello message and then expects to receive message A and then message B before replying with a Finished message.

Now assume that in the next TLS library version, the developer must implement an additional TLS cipher suite where – if this new suite was negotiated at the beginning of the TLS handshake – the TLS client expects to receive message C and then message B.

To reuse the message parsing and generation code for Hello and Finished, the developer might be tempted to modify the composite state machine so that, after receiving Hello, it can receive either A or C, followed by either B or D. While this simple composition implements both TLS cipher suites, it also allows unintended message sequences such as Hello, A, D, Finished.

Figure 22.1: Incorrect union of exemplary state machines

Intuitively, you might think that allowing unintended message sequences such as Hello, A, D, Finished does not effect security because Alice and Bob will send only valid message sequences and any attempt by Mallory to insert an incorrect message will be detected when Alice and Bob compare their communication transcripts.

This is, however, not the case. If Bob’s state machine accepts the sequence Hello, A, D, Finished, until he and Alice compare their transcripts, Bob is executing an unknown handshake protocol with no a priori security guarantees. As an example, the code for processing message D may be expected to always run after the code processing message C. Given Hello, A, D, Finished, D’s code may unintentionally use an uninitialized data structure that it expected to be filled by C’s code. Alternatively, it could leak secrets received in A or allow authentication steps to be skipped.

Using a combination of automated testing and manual source code analysis, the researchers conducted a systematic analysis of state machines implemented in then-current versions of open source TLS libraries [28]. While the individual state machines were correct, the researchers identified numerous TLS implementations that contained flaws in their composite state machines.

Some of these flaws had no effect on security. Others, however, turned out to be critical vulnerabilities that can be exploited by a remote attacker to undermine TLS security. More specifically, the researchers discovered several ways to impersonate a TLS server to a client running a flawed TLS implementation. One such attack, referred to as SKIP, simply skips the handshake messages. Another attack, which the researchers called FREAK, factors the server’s export-grade RSA key.