A byte stream does not know where your messages end
TCP delivers an ordered byte stream, so an application protocol must define message boundaries. A four-byte length prefix provides one simple framing rule, with explicit size limits and buffering for incomplete or coalesced frames.
The stream has no message edges
Suppose a sender writes the bytes for hello once. A receiver might read all five bytes together, receive only he in one read and llo in another, or read those bytes alongside the start of the next message. TCP preserves byte order, but its read boundaries do not mark application writes. The receiver sees a stream, not a queue of messages.
That distinction is the reason a protocol needs framing. HTTP defines messages at the application layer and, in HTTP/1.1, uses length-based delimiters for content among its framing mechanisms. The transport carries bytes; the protocol gives those bytes meaning and determines where each message ends. [2]
Choose a boundary rule
A fixed-width header followed by a payload is a compact framing scheme. Here the header is an unsigned four-byte integer in network byte order. Its value is the number of payload bytes to consume, so a frame consists of four header bytes followed by exactly that many bytes. The !I format in Python's struct module selects standard-size, big-endian representation, avoiding dependence on the machine's native byte order or alignment. [1]
For a five-byte payload, the frame is nine bytes total: four bytes encode the value five, followed by the bytes for hello. A decoder first waits for the complete header, then reads the declared size, then waits for the complete payload. If another frame follows immediately, the decoder returns those trailing bytes for the next parsing step.
import struct
LIMIT = 1024
def encode(payload):
if len(payload) > LIMIT:
raise ValueError("frame too large")
return struct.pack("!I", len(payload)) + payload
def decode(buffer):
if len(buffer) < 4:
return None
size = struct.unpack("!I", buffer[:4])[0]
if size > LIMIT:
raise ValueError("frame too large")
if len(buffer) < 4 + size:
return None
return buffer[4:4 + size], buffer[4 + size:]
assert decode(encode(b"hello")) == (b"hello", b"")
assert decode(encode(b"hello")[:6]) is None
Walk through one frame
The example encodes one payload and decodes from a buffer. Encoding rejects payloads over 1024 bytes before constructing the frame. Decoding returns None when it lacks a complete header or when the header is present but the payload has not all arrived. Once complete, it returns a pair: the payload and any unconsumed suffix. The first assertion describes a complete round trip; the second gives the decoder only six bytes, which contain the four-byte header and two payload bytes, so it must wait.
The decoder's size check occurs as soon as the header is available. That matters because a peer can declare a very large body without sending it; a parser should reject lengths outside its agreed bound before waiting or allocating based on them. The example bounds the payload at 1024 bytes. A real format would also specify behavior for malformed headers and decide how the caller retains and feeds incomplete data.
Buffering is part of parsing
Imagine two frames arrive in one read: the first is complete, and the second is complete too. The decoder returns the first payload plus the entire second frame as the remainder. The caller must parse that remainder again, potentially in a loop. Conversely, if a read ends halfway through a payload, the caller keeps the partial bytes and appends the next read before trying again. These are ordinary outcomes of stream delivery, not protocol errors.
A minimal decoder like this does not manage a socket, maintain a receive buffer, or handle a sequence of frames on its own. Those responsibilities belong to the surrounding code. Keeping framing separate from transport makes the rule easier to reason about: parse a header, validate its length, wait for the declared bytes, then leave the next frame untouched.
A small format makes assumptions visible
Length-prefix framing is easy to scan and can carry arbitrary binary payloads, including payloads containing zero bytes. Its tradeoff is that a corrupted or malicious length can disrupt parsing, which makes a strict maximum essential. The four-byte field also imposes a representable range, though a much lower protocol limit is usually more useful for controlling resource use.
Before implementing a framing rule, write down its byte order, header width, length meaning, maximum size, and incomplete-input behavior. Those details turn an informal exchange of bytes into a protocol a second implementation can follow.