Docs /architecture/ Payload Codecs
architecture / transport

Payload Codecs

Specialized payload codec design, Protobuf, Arrow, binary formats and selection.

codecsprotobufarrowtransportDocumentation latest

This is a detailed explanation of codec selection and extension. For the data model, see DataContainers; for an adapter walkthrough, see Extending the Protocol.

Sinapsis Protocol defines a universal transport contract around DataContainer.

The outer transport format is standardized so that agents can communicate using the same interface regardless of the types of data they exchange.

Individual payloads, however, do not need to use the same serialization format.

A payload can use the representation that is most appropriate for its data:

  • NumPy arrays → raw binary

  • Torch tensors → raw binary

  • DataFrames → Arrow

  • Files → artifacts

  • Raw bytes → MessagePack binary

  • Structured custom payloads → protobuf

  • Other application-specific types → registered codecs

This document describes that distinction and explains when a specialized payload codec is useful.


The Core Principle

The universal agent interface is always:

text
Agent
  │
  ▼
DataContainer
  │
  ├── standard payload
  ├── tensor
  ├── image
  ├── dataframe
  ├── file
  └── custom payload

The DataContainer is the protocol contract.

A specialized codec does not replace DataContainer.

Instead, it defines how one particular payload is represented inside the container:

text
DataContainer
      │
      ▼
    Packet
      │
      ▼
    content
      │
      ▼
specialized codec
      │
      ├── raw binary
      ├── Arrow
      ├── protobuf
      └── application-specific encoding

This means an agent can always expect a DataContainer, even when the contents use highly specialized serialization formats.


Why Use a Specialized Payload Codec?

MessagePack is an excellent general-purpose envelope, but it is not necessarily the most efficient representation for every kind of structured data.

A specialized codec can provide:

  • smaller payloads

  • faster serialization

  • faster deserialization

  • strongly defined schemas

  • language interoperability

  • generated types

  • efficient representation of structured data

  • better compatibility with an existing ecosystem

For example, a custom structured message may be much more naturally represented by protobuf than by a generic MessagePack dictionary.

The protocol therefore separates:

text
Transport
    │
    ▼
DataContainer + MessagePack

Payload representation
    │
    ▼
Application-specific codec

Protobuf as a Payload Codec

Protobuf is useful when a payload has a well-defined, structured schema and needs efficient binary serialization or cross-language interoperability.

For example:

text
message DetectionResult {
    string label = 1;
    float confidence = 2;
    repeated float bbox = 3;
}

The generated Python object can be used as the content of a packet:

text
from dataclasses import dataclass

from my_proto_pb2 import DetectionResult


@dataclass
class DetectionPacket:
    content: DetectionResult
    source: str | None = None

The resulting protocol structure is:

text
DataContainer
└── DetectionPacket
    └── content
        └── DetectionResult
            └── protobuf bytes

The outer transport remains unchanged.


Registering a Protobuf adapter

The exact adapter interface is implementation-specific. For sinapsis-protocol 0.1.0, use the versioned extension guide and its registered leaf-adapter example rather than copying the older, illustrative adapter implementation.


Does MessagePack Defeat Protobuf?

No.

The protobuf payload is stored as a binary value inside MessagePack.

The protobuf bytes are not converted into a MessagePack representation.

The flow is:

text
DetectionResult
      │
      ▼
protobuf.SerializeToString()
      │
      ▼
     bytes
      │
      ▼
MessagePack binary field

There is no:

text
protobuf → dictionary → MessagePack

conversion.

Instead:

text
protobuf → bytes → MessagePack binary field

MessagePack is responsible for the surrounding protocol structure.

Protobuf remains responsible for the payload.


Performance Considerations

A specialized payload codec is primarily useful when the payload itself is large enough or structured enough for the codec's benefits to matter.

For example:

text
Small payload

DataContainer
└── 200-byte protobuf message

The MessagePack envelope may represent a meaningful fraction of the total packet.

For a large payload:

text
Large payload

DataContainer
└── 10 MB protobuf message

the additional MessagePack overhead is comparatively small because MessagePack is simply carrying the protobuf byte string.

The expensive operation remains:

text
Application object
      │
      ▼
   protobuf
      │
      ▼
large binary payload

MessagePack then adds the surrounding transport metadata.

For large payloads, the benefits of a specialized codec can therefore outweigh the small cost of the universal envelope.


What Specialized Codecs Are For

A specialized codec is appropriate when the payload has characteristics that justify a specialized representation.

Protobuf

Good for:

  • structured messages

  • explicit schemas

  • cross-language communication

  • generated classes

  • long-lived message contracts

    Custom object │ ▼ protobuf │ ▼ binary payload

Arrow

Good for:

  • tabular data

  • dataframe interchange

  • columnar data

  • cross-language dataframe processing

    DataFrame │ ▼ Arrow IPC │ ▼ binary payload

Raw Binary

Good for:

  • tensors

  • images

  • audio

  • arbitrary byte streams

  • large numerical arrays

    NumPy / Torch / bytes │ ▼ binary representation │ ▼ binary payload

Artifact Storage

Good for very large objects that should not be embedded directly in the transport packet.

text
Large object
    │
    ▼
ArtifactStore
    │
    ▼
ArtifactRef
    │
    ▼
DataContainer

What Specialized Codecs Are Not

A specialized codec is not a replacement for the Sinapsis transport protocol.

It should not introduce a second agent-level transport contract such as:

text
Agent A
   │
   ▼
protobuf
   │
   ▼
Agent B

when the agent interface is expected to use DataContainer.

Instead:

text
Agent A
   │
   ▼
DataContainer
   │
   └── protobuf payload
            │
            ▼
         Agent B

This keeps the agent interface standardized.

An agent should not need to know whether the payload was encoded using:

  • NumPy binary

  • Torch binary

  • Arrow

  • protobuf

  • another registered codec

in order to understand the transport contract.


What Should Not Be Done

Do not introduce unnecessary codec layers.

For example, this is usually undesirable:

text
NumPy
  │
  ▼
JSON
  │
  ▼
protobuf
  │
  ▼
MessagePack

Each additional representation can add:

  • CPU work

  • memory copies

  • latency

  • packet size

  • implementation complexity

Prefer the shortest appropriate representation:

text
NumPy
  │
  ▼
binary

or:

text
Custom structured object
      │
      ▼
   protobuf
      │
      ▼
   binary

or:

text
DataFrame
    │
    ▼
  Arrow
    │
    ▼
  binary

Large Payloads

Specialized codecs are particularly useful for large payloads.

A typical flow is:

text
                    DataContainer
                         │
                         ▼
                      Packet
                         │
                         ▼
                      content
                         │
              ┌──────────┴──────────┐
              │                     │
         small payload        large payload
              │                     │
              ▼                     ▼
        inline binary         ArtifactStore
                                    │
                                    ▼
                               ArtifactRef

The protocol can therefore choose between:

text
inline

and:

text
artifact

without changing the agent-facing DataContainer contract.


Choosing a Codec

The general rule is:

Use the simplest representation that efficiently and safely represents the payload.

A useful decision process is:

◇ DiagramDrag nodes · Scroll to zoom · Drag canvas to pan
flowchart TD
    INPUT([Payload to encode]) --> SELECT{Payload characteristics}

    SELECT -->|Already bytes| B[Use binary]
    SELECT -->|Array or tensor| N[Use native binary]
    SELECT -->|Tabular data| A[Consider Arrow]
    SELECT -->|Structured schema| P[Consider Protobuf]
    SELECT -->|Other| C[Standard representation<br/>or registered codec]

    B --> OUTPUT([Encoded payload])
    N --> OUTPUT
    A --> OUTPUT
    P --> OUTPUT
    C --> OUTPUT

Green-bordered nodes reference Sinapsis templates. Hover or focus them to fetch documentation.

View Mermaid source
mermaid
flowchart TD
    INPUT([Payload to encode]) --> SELECT{Payload characteristics}

    SELECT -->|Already bytes| B[Use binary]
    SELECT -->|Array or tensor| N[Use native binary]
    SELECT -->|Tabular data| A[Consider Arrow]
    SELECT -->|Structured schema| P[Consider Protobuf]
    SELECT -->|Other| C[Standard representation<br/>or registered codec]

    B --> OUTPUT([Encoded payload])
    N --> OUTPUT
    A --> OUTPUT
    P --> OUTPUT
    C --> OUTPUT

The goal is not to use protobuf everywhere.

The goal is to allow the protocol to use protobuf where protobuf provides a meaningful advantage.


Protocol Boundary

The architectural boundary is:

text
┌───────────────────────────────────────────────┐
│              Sinapsis Protocol                │
│                                               │
│  DataContainer                                │
│       │                                       │
│       ├── packet metadata                     │
│       ├── packet structure                    │
│       └── payload descriptor                  │
│                                               │
│       └──────────────┐                        │
│                      ▼                        │
│               Payload Codec                   │
│                      │                        │
│          ┌───────────┼───────────┐            │
│          ▼           ▼           ▼            │
│       binary       Arrow      protobuf        │
│                                               │
└───────────────────────────────────────────────┘

The outer boundary remains stable.

Payload codecs are extensibility points inside that boundary.


Summary

The design intentionally separates transport standardization from payload serialization.

The universal contract is:

text
DataContainer

The universal wire envelope is:

text
MessagePack

The payload representation is extensible:

text
MessagePack
    │
    └── payload
          ├── binary
          ├── Arrow
          ├── protobuf
          ├── artifact reference
          └── custom registered codec

This provides a stable interface for agents while allowing individual payloads to use specialized serialization when it provides better performance, interoperability, or schema management.

The key principle is:

Standardize the container. Optimize the payload.

Published from Git revision 7b7ba25d627b