Lecture 15: Authenticated Encryption

Announcements

Learning outcomes

By the end of today’s lecture, you will learn:

  • The role that pseudorandomness plays in symmetric cryptography
  • How to construct symmetric key encryption, message authentication codes, and authenticated encryption using a sponge function design

Review from last lecture

“Cryptography is about communication in the presence of an adversary.”

– Prof. Ron Rivest

Encryption is one of the main tools that we use to protect data confidentiality. The goal of Part 3 of this course is to study encryption and its role in:

  • Protecting data at rest on your laptop or phone
  • Protecting data in transit over the Internet or messaging app

Encryption is a powerful tool with immense social consequences.

“Cryptography rearranges power: it configures who can do what, from what.”

– Prof. Phillip Rogaway (UC Davis)

Internet communication, with encryption

For this reason, I encourage you to put yourself in the role of Mallory and ask “what if” questions.

  • What if Mallory steals Bob’s secret key (for public key encryption)?
  • What if Mallory temporarily compromises Bob’s phone or laptop (for symmetric encryption)?

In particular, encryption has an all-or-nothing property: anyone who possesses the secret key can decrypt and read a message. It does not matter whether or not you wanted them to!

News article in The Atlantic (link)

A symmetric key encryption scheme has three methods: KeyGen, Enc, and Dec. It satisfies two properties.

  1. Correctness: Dec undoes the action of Enc. That is, for any key \(k\), message \(P\), and randomness \(R\) it holds that \[\text{Dec}(k, \text{Enc}(k, P; R)) = P.\]

  2. Secrecy (informally): If an attacker Mallory sees the ciphertext \(C\) but not the symmetric key \(k\), then she has no chance of learning any part of the plaintext message \(P\).

Question. How can Alice encode messages so Mallory cannot read them?

Option 1: substitute each character (source: Wikipedia)

Option 2: substitute each word or phrase

Last time, we discussed the idea of a one-time pad, which satisfies perfect secrecy but has several downsides.

  1. The symmetric key \(k\) must be as long as the plaintext message \(P\).
  2. Reusing the same key for two or more messages breaks perfect security, and brings back the cryptograms issue.
  3. The cipher is malleable! While OTP is safe against a passive eavesdropper Eve, an active attacker Mallory can manipulate the ciphertext in transit and cause the same change in the decrypted plaintext.

15.1 Pseudorandomness

What if we want to encrypt a long message using a short key?

  • It turns out that perfect secrecy is impossible.
  • But that’s okay; we can resort back to computational hardness (e.g., “energy of the sun” work required to break the scheme)

Our goal for today is to construct two kinds of symmetric key primitives.

  • Symmetric key encryption allows two parties to send messages to each other with confidentiality. Each party can both encrypt and decrypt.

  • A message authentication code allows two parties to send messages to each other with integrity. It is the symmetric-key version of a signature scheme, where each party can both sign and verify.

These primitives are nearly always used together. So for convenience, we will construct a combined primitive called authenticated encryption that performs both of these tasks at once.

“Confidentiality xor authenticity is not possible.
If you don’t have both, often you don’t have either.”

– Prof. Matthew Green (Johns Hopkins)

A new kind of random oracle

Modern symmetric encryption schemes are designed using a pseudorandom permutation \(f: \{0, 1\}^n \to \{0, 1\}^n\). This function is deterministic but “acts random.”

A pseudorandom permutation (or PRP) is almost like a cryptographic hash function, except that:

  • The input and output lengths are equal.
  • There are no collisions: the function is a permutation, so each output occurs exactly once.

As a result, this primitive is somewhat easier to build, and therefore faster, than a cryptographic hash function.

Still though, we require that the truth table (that maps inputs to outputs) “appears random.”

In other words, the pseudorandom permutation acts as a “foreign language” that removes all structure from the (say) English-language input… except that using the exact same input twice will result in the same output.

Option 2: substitute each word or phrase

Advanced Encryption Standard

The most common pseudorandom permutation used in practice is the Advaned Encryption Standard (AES).

  • It was designed by Joan Daemen and Vincent Rijmen in the 1990s
  • It won a NIST competition (among 15 total entries) to be named as the pseudorandom permutation used by the U.S. federal government.

“This Standard may be used by federal agencies to protect information when they have determined that encryption is appropriate.”

– NIST Federal Information Processing Standards (FIPS) publication 197, “Announcing the Advanced Encryption Standard (AES)” (link)

In fact, AES is a family of many pseudorandom permutations, indexed by a key. To anyone who does not know the key, the function acts like a random oracle.

If an adversary [Mallory] has not explicitly queried the oracle on some point \(x\), then the value of \(H(x)\) is completely random… at least as far as [Mallory] is concerned.

– Jon Katz and Yehuda Lindell, Introduction to Modern Cryptography

I show below an example of the AES permutation (for a randomly-chosen key). Note that the function has a fixed input and output length of 16 bytes. Unlike a hash function, it does not work for any other size.

from Crypto.Cipher import AES
from binascii import hexlify, unhexlify
from os import urandom
key = urandom(16)

def permutation(x):
    cipher = AES.new(key, AES.MODE_ECB)
    return hexlify(cipher.encrypt(unhexlify(x)))

try:
    print(permutation(b'00' * 15))
except:
    print("Input too short")
Input too short
try:
    print(permutation(b'00' * 17))
except:
    print("Input too long")
Input too long

As one would expect from a random-looking function: making small changes to the input results in substantial changes to the output.

print(permutation(b'00' * 16))
print(permutation(b'00' * 15 + b'01'))
b'aed2f463e960198b11c05ae30a96e4d8'
b'a13339b8c43d1283dd4e9a371d6bb93d'

Question. Which of the following bitstrings “looks” random and unpredictable?

  • 11111111
  • 01010101
  • 10100011

Answer. This is actually a trick question: if the strings were sampled uniformly at random, then each output is equally likely!

This brings us to an important insight:

  • You cannot look at a single output string and determine its (un)predictability
  • Our pseudorandomness definition instead evaluates the process of designing a pseudorandom function. This requires cryptanalysis and relies on Schneier’s law.

“Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can’t break.”

– Bruce Schneier

15.2 Sponge functions

Today, we will look at the inner workings of a newer NIST standard: the SHA-3 cryptographic hash function.

  • NIST ran a competition from 2007 to 2012 to choose a ‘successor’ hash function to SHA-2.
  • They received 64 submissions, and through annual workshops and lots of cryptanalysis, they slowly cut down the number of competitors: 64 \(\to\) 51 \(\to\) 14 \(\to\) 5 \(\to\) 1.
  • In October 2012, they announced the Keccak algorithm as the winner.
  • Its creators are Guido Bertoni, Joan Daemen, Michaël Peeters, and Gilles Van Assche

Why NIST chose Keccak, in their words:

  1. “Offers acceptable performance in software, and
excellent performance in hardware.”
  2. “Has a large security margin, suggesting a good chance of surviving without a practical attack during its working lifetime.”
  3. “A fundamentally new and different algorithm that is entirely unrelated to the SHA-2 algorithms.”

The notion of security margins is also related to Schneier’s law.

The design of Keccak contains two main components.

  1. Fixed-length permutation: a deterministic, pseudorandom permutation called Keccak-\(f\) with a fixed input/output length of \(b = 1600\) bits, which is carefully designed to act like a truly random function.

  2. Mode of operation: an interative way to make many calls to the \(f\) function in order to process each chunk of a variable-length input.

The mode used by Keccak is called a sponge function. It has two stages.

  • It first absorbs an arbitrary-length input.
  • It then squeezes an arbitrary-length output.

Sponge function design (image source: Wikipedia)

At any given time, the sponge function remembers \(b\) bits of state. It cleverly partitions the state into two components.

  • The first \(r\) bits are called the rate. This part determines the throughput of the sponge function, i.e., the number of bits of the message that get processed by each call to the Keccak-\(f\) function.

  • The remaining \(c\) bits are called the capacity. This part determines the security of the sponge function, up to a birthday bound attack (i.e., about \(2^{c/2}\) work is required to break it).

“When for instance \(c = 512\), a random sponge offers the same resistance as a random oracle but with a maximum of \(2^{256}\) in complexity.”

– Bertoni, Daemen, Peeters, and Van Assche. On the Indifferentiability of the Sponge Construction

Hashing

When used as a cryptographic hash function, Keccak is instantiated with a fixed output length (e.g., 224, 256, 384, or 512 bits).

Sponge function design (image source: Keccak presentation to NIST)

Note that the message only touches the rate bits of the state. Meanwhile, the capacity bits are only influenced by the Keccak-\(f\) function itself, and therefore always remain “random-looking.”

By a birthday bound, two states with random-looking capacity bits are unlikely to collide.

Salted hashing

In many applications that use hash functions, it is common to add a nonce to the input. This random, public value is used to:

  • Ensure that hashes in your application are different than hashes in other applications, and
  • Prevent pre-computation of the hashes of popular messages.

In hash function lingo, this nonce is typically called a salt.

Salted hash function (image source: Keccak presentation to NIST)

Password hashing

Have you ever wondered how your computer stores the password to log into itself? It seems like circular logic if:

  • The computer grants access to anyone sitting in front of it who knows the correct password \(p\).
  • The computer stores the login password \(p\) so it can check whether you have typed it in correctly.

For this reason:

  • Computers (and web servers) store a hash of the true password \(h(p)\) rather than the cleartext password.
  • After you type a password \(p'\), the computer hashes it and compares against the stored hash.

Remember the preimage resistance guarantee provided by cryptographic hash functions: given a random \(y\), it is difficult to find any preimage \(x\) such that \(H(x) = y\).

However, passwords are generated by people. As a result, they are not chosen uniformly at random, and in practice they tend to follow Zipf’s law.

  • Some passwords are extremely common and guessable, like 123456 or password.

  • Many passwords are short and brute-forceable.

  • Some people choose long, strong passwords.

As a result, password hash functions are specifically designed to be slow and resource-consuming.

  • When executed once by the honest account-holder, a delay even in the 0.1-1 second range is perhaps tolerable.
  • But for an attacker Mallory who tries to brute-force guess millions or billions of passowrds, the delays add up.

Question. How do you make a function deliberately slow, in a way that is impossible to speed up?

Keccak’s sponge function design provides an elegant way to design a computationally-slow hash function: simply absorb more data.

Sponge function design (image source: Keccak presentation to NIST)

Newer password hash functions like argon2 are also designed to require a lot of memory, so that a brute-force attack is harder to execute on a GPU or dedicated hardware device.

Symmetric encryption

Recall that two of the problems with a one-time pad were key length and key reuse.

  • The symmetric key must be as long as the plaintext message.
  • Reusing the same key for two or more messages breaks perfect security, and brings back the cryptograms issue.

We can use a sponge function to address these issues. Given only:

  • A short cryptographic key, and
  • A nonce called an “initialization vector” (IV),

we can pseudorandomly build a long key stream that Alice and Bob can XOR with their private data.

Making a one-time-use synthetic key (image source: Keccak presentation to NIST)

It is imperative that each IV be used only once. If an IV is reused, then we are back to the two-time pad problem. That said, the IV does not need to be kept hidden from the adversary.

In other words:

  • The key is used to provide uncertainty.
  • The IV is used to provide variety.

15.3 Message authentication

Pseudorandomness can be used to build a message authentication code (MAC), which allow two people on the Internet to know that they are talking to each other.

  • Note that this is an integrity guarantee, not a confidentiality guarantee.
  • MACs addresses the third issue with a one-time pad: that the cipher was malleable.
         message  0110 0110 1001               cipher   0101 1010 1000
     XOR key      0011 1100 1110           XOR key      0011 1100 1110
         -----------------------               -----------------------
         cipher   0101 1010 0111               message  0110 0110 0110

MACs are also precisely what we need in order to instantiate a key building block that we needed in the phase-king and Bracha reliable broadcast protocols.

Authenticated pair-wise channels: Every party is connected to every other party, and they somehow have a way to know who is communicating with them. In other words: no party can impersonate another, so a party Bob can be confident that he is receiving messages from another party Alice.

– This course, Lecture 9

Defining MACs

A MAC is the symmetric-key version of a digital signature. As a result, it combines:

  • The KeyGen algorithm from symmetric key encryption.
  • The MAC and Verify algorithms that operate similarly to their signature counterparts.

Formally, suppose there is a message \(A\) that needs to be authenticated in transit over the Internet.

Definition. A message authentication code contains three algorithms:

  • KeyGen: Alice and Bob jointly sample a symmetric key \(k\) from a distribution \(\mathcal{K}\) (say, a uniformly random bytestring of fixed length)
  • MAC\((k, A)\): produces a tag \(T\)
  • Verify\((k, A, T)\): returns true or false

All three algorithms should be efficiently computable. Moreover, the digital signature must satisfy two requirements:

  • Correctness, meaning that validly-produced MAC tags will always successfully verify.
  • Existential unforgeability under a chosen message attack (EU-CMA), which is defined in exactly the same way as for public-key digital signatures.

Note that the same key is used in both MAC and Verify. As a result:

  • With a digital signature scheme, the entire world can verify the identity of the message sender.
  • With a MAC, only the message receiver (who also holds the symmetric key) verify the sender’s identity.

MACs for short messages

If the authenticated message \(A\) is short, then there is a simple way to construct a message authentication code: apply a pseudorandom permutation on the key and message: MAC\((k, A) = f(k, A)\).

  • The intuitive idea here is that the attacker Mallory has no idea how to forge the “random-looking” outputs that come from \(f\).

  • Formally reasoning about this claim is somewhat complicated. It requires a connection between two different types of security guarantees, in order to form a reduction between the two claims (in the style of complexity theory).

Theorem. If there exists an adversary \(\mathcal{A}\) that forges the MAC, then it is possible to construct an adversary \(\mathcal{B}\) that can distinguish between pseudorandom and truly random permutations.

Question. Why is this theorem helpful to us? (At first glance, it sounds like it is helping Mallory: it provides her with a roadmap for how to break things.)

MACs for variable length messages

Keccak-based MAC (image source: Keccak presentation to NIST)

The Keccak sponge function construction can be used to design a MAC for variable length messages.

  • We can absorb a long message (with padding to ensure that we have an integer multiple of the rate).
  • Then we can squeeze a pseudorandom output.

15.4 Authenticated encryption

Because symmetric key encryption and MACs are so useful (and fast!), it is common to use them together. That is: for most messages that we send over the Internet, we want data confidentiality and integrity!

Question. Suppose there is a message \(P\) that we want to be private and authenticated. What is the best way to combine encryption and MACs?

Generically combining encryption with MACs

If you don’t know anything about the specific symmetric key encryption and MAC constructions used, then the most important thing to remember is that the MAC does not provide data confidentiality.

This leads us to the right answer for this question, and more generally to the cryptographic doom principle.

“If you have to perform any cryptographic operation before verifying the MAC on a message you’ve received, it will somehow inevitably lead to doom!”

– Moxie Marlinspike

If you do know the specific encryption and MAC used, then it is possible to combine them in a more efficient manner. For instance, once again we can use the Keccak sponge function design to build a combined Enc+MAC.

Authenticated encryption with a sponge function (image source: Keccak presentation to NIST)

Defining AEAD

This combined primitive is called authenticated encryption. In general, suppose that the sender Alice has:

  • Data \(P\) that she wants to be private and authenticated,
  • Associated data \(A\) that she only wants to be authenticated, and
  • Nonce \(N\) (sometimes also called the initialization vector).

Definition. Authenticated encryption with associated data (AEAD) contains three algorithms:

  • KeyGen: randomly choose \(k\), as usual
  • Enc\((k, P, A, N)\): produces a ciphertext/tag \(C\) of length \(|C| \geq |P|\)
  • Dec\((k, C, A, N)\): returns the original private message \(P\)

The AEAD security game combines aspects of ciphertext indistinguishability and tag unforgeability. The simplest way to explain it is via the following thought experiment: Mallory cannot tell the difference between the two sides of this picture.

AEAD security game

Why should we combine authentication and encryption?

  • Better security: satisfies Moxie’s cryptographic doom principle
  • Simplicity: developers have fewer decisions (i.e., opportunities for mistakes)
  • Performance: save in time + space costs, also often only need 1 key

Next time, we will apply authenticated encryption in order to protect data at rest on your hard drive.