Programming · free software · the open webSubscribe via RSS

A fast wire layout still needs a trust boundary

Cap’n Proto’s word-aligned layout supports efficient traversal, but a receiver still has to validate bounds and resource use. Serialization also says nothing by itself about who sent a message or what that sender may do.

A message is a graph-shaped layout with a tree-shaped rule

Cap’n Proto messages are organized as segments: flat regions of bytes made up of 64-bit words. The first word of the first segment points to the root struct. A struct pointer describes where the struct’s content begins and how many words belong to its data and pointer sections. Those sections are adjacent, with the pointer section after the data, so generic traversal can walk a struct’s slots without knowing the schema’s field meanings. The schema still matters when application code interprets those slots. [1]

Pointers lead to objects such as structs and lists. The encoding’s object model requires pointers to target object beginnings, with no more than one pointer to an object; the resulting objects and references form a tree. A list pointer records its element representation and extent. Composite lists use a tag describing each element’s struct shape, while the pointer records the list’s total word extent. These rules make layout regular, but they do not make every arbitrary byte sequence a valid message. [1]

Segments trade relocation for framing work

A single segment is convenient when a message fits in one contiguous allocation. Multiple segments let a writer grow a message without relocating bytes already written, or avoid reserving an oversized block up front. References that cross segment boundaries use far pointers and landing pads rather than ordinary relative offsets. A receiver therefore needs the segment table and lengths before it can safely follow references. The encoding specification leaves stream framing to the transport: the receiver must learn how many segments there are and how large they are. [1]

That framing data is part of the input boundary. A parser must reject a segment identifier outside the supplied table, an offset that points beyond a segment, and a list extent whose element count or word count exceeds available storage. It should also impose traversal and allocation budgets. A message can fit in the received bytes yet describe work that is too expensive for the application to process; a limit on bytes alone does not cap all consequences of traversing pointers or constructing application objects.

Zero-copy is conditional, not magic

Aligned words and relative pointers can allow a reader to access message data directly in the segment buffer, avoiding a separate object-by-object decode. That is the useful sense of zero-copy: it depends on the buffer remaining available, correct alignment and bounds handling, and an API that can expose borrowed views safely. If a field must outlive the input buffer, cross an ownership boundary, or be converted into a different representation, copying may still be necessary.

Packed bytes are a separate optimization. The encoding describes a simple packing scheme that compresses zero-value bytes. A packed stream must be unpacked before ordinary message traversal; packing does not turn a byte stream into directly traversable segments. In either representation, the decoder must establish valid segment boundaries and pointer targets before treating the data as a message. [1]

Layout does not grant authority

Suppose a service receives a well-formed message whose root asks to delete a document. Successful bounds checks establish that the request can be read safely; they do not establish that the sender is allowed to delete that document. Authentication identifies or verifies a peer through a separate mechanism. Transport encryption can protect a connection in transit, but the RPC documentation says Cap’n Proto does not specify an encryption scheme and can be layered over TLS. Application authorization must still decide whether this caller may perform this operation on this object. [2]

Cap’n Proto RPC can convey capabilities: references that designate callable objects and confer permission to call them. That is an authority model above the serialized layout, with its own protocol and application decisions. Treat the layers separately: frame and validate the bytes, constrain traversal and resource consumption, secure the transport where needed, then enforce the application’s access rules. A fast wire representation helps with data movement; it cannot answer the trust questions for you. [1] [2]

References

  1. Cap’n Proto encoding
  2. Cap’n Proto RPC protocol

Suggest a correction · All posts