CVE-2023-36478

An integer overflow in MetaDataBuilder.checkSize allows for HTTP/2 HPACK header values to
exceed their size limit.

In MetaDataBuilder.java, the following code determines if a header name or value
exceeds the size limit, and throws an exception if the limit is exceeded:

291 public void checkSize(int length, boolean huffman) throws SessionException
292 {
293 // Apply a huffman fudge factor
294 if (huffman)
295 length = (length * 4) / 3;
296 if ((_size + length) > _maxSize)
297 throw new HpackException.SessionException("Header too large %d > %d",
_size + length, _maxSize);
298 }

However, when length is very large and huffman is true, the multiplication by 4 in line 295
will overflow, and length will become negative. (_size+length) will now be negative, and
the check on line 296 will not be triggered.

Furthermore, MetaDataBuilder.checkSize allows for user-entered HPACK header value sizes to be
negative, potentially leading to a very large buffer allocation later on when the
user-entered size is multiplied by 2.

In MetaDataBuilder.java, the following code determines if a header name or value
exceeds the size limit, and throws an exception if the limit is exceeded:

public void checkSize(int length, boolean huffman) throws SessionException
{
// Apply a huffman fudge factor
if (huffman)
length = (length * 4) / 3;
if ((_size + length) > _maxSize)
throw new HpackException.SessionException("Header too large %d > %d", _size
+ length, _maxSize);
}

However, no exception is thrown in the case of a negative size.
Later, in Huffman.decode, the user-entered length is multiplied by 2 before allocating a buffer:

public static String decode(ByteBuffer buffer, int length) throws
HpackException.CompressionException
{
Utf8StringBuilder utf8 = new Utf8StringBuilder(length * 2);
// ...

This means that if a user provides a negative length value (or, more precisely, a length
value which, when multiplied by the 4/3 fudge factor, is negative), and this length value is a
very large positive number when multiplied by 2, then the user can cause a very large
buffer to be allocated on the server.

Exploit Scenario 1

An attacker repeatedly sends HTTP messages with the HPACK header 0x00ffffffffff02.
Each time this header is decoded:

• HpackDecode.decode will determine that a Huffman-coded value of length
  805306494 needs to be decoded.
• MetaDataBuilder.checkSize will approve this length.
• Huffman.decode will allocate a 1.6 GB string array.
• Huffman.decode will have a buffer overflow error, and the array will be deallocated
  the next time garbage collection happens. (Note: this can be delayed by appending
  valid huffman-coded characters to the end of the header.)

Depending on the timing of garbage collection, the number of threads, and the amount of
memory available on the server, this may cause the server to run out of memory.

Exploit Scenario 2

An attacker repeatedly sends HTTP messages with the HPACK header 0x00ff8080ffff0b. Each
time this header is decoded:

• HpackDecode.decode will determine that a Huffman-coded value of length
  -1073758081 needs to be decoded
• MetaDataBuilder.checkSize will approve this length
• The number will be multiplied by 2 to get 2147451134, and Huffman.decode will
  allocate a 2.1 GB string array
• Huffman.decode will have a buffer overflow error, and the array will be deallocated
  the next time garbage collection happens (Note that this deallocation can be
  delayed by adding valid Huffman-coded characters to the end of the header)

Depending on the timing of garbage collection, the number of threads, and the amount of
memory available on the server, this may cause the server to run out of memory.

Workarounds

No workarounds possible, only patched versions of Jetty.

References

• https://github.com/eclipse/jetty.project/pull/9634