Classifying and decoding compressed HTML

1. Start with the right mental model

PHP's gzcompress() produces a zlib stream, not a gzip stream. With the default compression level, the zlib header is 78 9C. To make that stream compatible with MySQL's UNCOMPRESS(), the application prepends a four-byte, little-endian length of the uncompressed content.

Here, 89 06 00 00 is little-endian 0x00000689, or 1,673 bytes. The trouble begins when these arbitrary binary bytes cross a MySQL latin1 connection into an utf8mb4 character column.

MySQL maps original byte 89 to Unicode U+2030 (‰), whose UTF-8 encoding is E2 80 B0. It maps byte 9C to U+0153 (œ), encoded as C5 93. The logical four-byte prefix now occupies six physical bytes.

2. Recognize the four possible formats

3. Classify using header bytes only

Do not speculatively decompress the entire value. Check prefixed forms first, because the four arbitrary length bytes could coincidentally begin with zlib-like bytes.

To locate the double-encoded prefixed marker, walk exactly four canonical UTF-8 sequences from the beginning. Those four Unicode characters represent the original four single-byte length values. The resulting byte offset is between 4 and 12.

private function encodedPrefixLength(string $value): ?int
{
    $offset = 0;

    for ($character = 0; $character < 4; ++$character) {
        $sequenceLength = $this->mysqlLatin1Utf8SequenceLength($value, $offset);

        if (null === $sequenceLength) {
            return null;
        }

        $offset += $sequenceLength;
    }

    return $offset;
}

4. Reverse MySQL’s exact latin1 mapping

For either historical format, decode the entire value. This is not ordinary ISO-8859-1 conversion. MySQL's latin1 is based on CP1252 and assigns a Unicode character to every possible byte, including CP1252's five gaps.

private const UNICODE_TO_MYSQL_LATIN1 = [
    0x20ac => 0x80, 0x0081 => 0x81, 0x201a => 0x82, 0x0192 => 0x83,
    0x201e => 0x84, 0x2026 => 0x85, 0x2020 => 0x86, 0x2021 => 0x87,
    0x02c6 => 0x88, 0x2030 => 0x89, 0x0160 => 0x8a, 0x2039 => 0x8b,
    0x0152 => 0x8c, 0x008d => 0x8d, 0x017d => 0x8e, 0x008f => 0x8f,
    0x0090 => 0x90, 0x2018 => 0x91, 0x2019 => 0x92, 0x201c => 0x93,
    0x201d => 0x94, 0x2022 => 0x95, 0x2013 => 0x96, 0x2014 => 0x97,
    0x02dc => 0x98, 0x2122 => 0x99, 0x0161 => 0x9a, 0x203a => 0x9b,
    0x0153 => 0x9c, 0x009d => 0x9d, 0x017e => 0x9e, 0x0178 => 0x9f,
];

5. Normalize, decompress once, and validate

Classification determines which normalization is required. After normalization, every candidate is either a naked zlib stream or the canonical four-byte prefix followed by zlib.

private function decompressPrefixed(string $value): string|false
{
    if (strlen($value) < 6) {
        return false;
    }

    $expectedLength = unpack('Vlength', substr($value, 0, 4))['length'];
    $decompressed = @gzuncompress(substr($value, 4));

    if (false === $decompressed || strlen($decompressed) !== $expectedLength) {
        return false;
    }

    return $decompressed;
}

Use V, not L, for the length header. V explicitly means an unsigned 32-bit little-endian value, matching MySQL's format on every PHP host.

6. Put the shim together

public function convertToPHPValue($value, AbstractPlatform $platform): mixed
{
    if (null === $value) {
        return null;
    }

    $format = $this->classifyCompressionFormat($value);

    if ($format->isDoubleEncoded()) {
        $decoded = $this->reverseMysqlLatin1Encoding($value);

        if (false === $decoded) {
            return parent::convertToPHPValue($value, $platform);
        }

        $value = $decoded;
    }

    $decompressed = $format->hasLengthPrefix()
        ? $this->decompressPrefixed($value)
        : @gzuncompress($value);

    return parent::convertToPHPValue(
        false !== $decompressed ? $decompressed : $value,
        $platform,
    );
}

public function convertToDatabaseValue($value, AbstractPlatform $platform): mixed
{
    $data = parent::convertToDatabaseValue($value, $platform);

    if (null === $data) {
        return null;
    }

    return pack('V', strlen($data)) . gzcompress($data);
}

Recommended unit-test matrix

• Raw prefixed value returns its original content and validates the declared length.
• Raw naked value remains readable for legacy compatibility.
• Double-encoded prefixed value with a one-, two-, and three-byte encoded length character is recovered.
• Double-encoded naked value is recovered.
• A payload containing MySQL's special gap byte, such as 8F, round-trips through the custom decoder.
• Malformed UTF-8, an unmappable Unicode character, bad zlib data, and a wrong declared length fail safely.
• New writes use pack('V'), include the prefix, and can be read back.
• Valid decompressed values "" and "0" are not mistaken for failure.