Wire Format and Exchange Envelopes
MessagePack versioning, wire framing, SSE exchange envelopes and resource limits.
This article documents the concrete sinapsis-protocol 0.1.0 representation. The broader UDT Architecture is a protocol design layer rather than a separate serialization library.
Why this exists
Sinapsis data moves between agents, processes, workers, and services. A useful wire format needs to be:
recursive: a
DataContainercan contain packet dataclasses containing other structured values;versioned: protocol and transport types have explicit versions;
binary-friendly: arrays and other large payloads are transferred as bytes rather than JSON lists;
extensible: applications can register their own packet types and payload codecs;
artifact-aware: large payloads can live outside the MessagePack wire representation;
bounded: decoding applies configurable resource limits before large allocations;
safe by default: there is no automatic pickle/cloudpickle fallback;
dependency-light: optional libraries are isolated behind adapters;
replaceable: payload representations can be specialized without changing the universal
DataContainerboundary.
Wire format
Normal packets use a MessagePack envelope:
{
"protocol": "sinapsis-data",
"major": 1,
"minor": 0,
"kind": "container",
"payload": ...,
}
A dataclass node is represented conceptually as:
{
"__type__": "dataclass",
"version": 1,
"name": "stable-wire-name",
"fields": {
"field_a": ...,
"field_b": ...,
},
}
Protocol metadata and payload descriptions are represented internally by strongly typed protocol models. Dictionaries are used at the final wire-format boundary.
Conceptually, the payload model is:
from sinapsis.data_transport import dataclass
from sinapsis_protocol.wire.constants import PayloadCodec, PayloadKind
@dataclass(frozen=True)
class PayloadDescriptor:
kind: PayloadKind
codec: PayloadCodec
version: int = 1
@dataclass(frozen=True)
class InlinePayload:
data: bytes
sha256: str | None = None
@dataclass(frozen=True)
class ArtifactPayload:
artifact: ArtifactRef
@dataclass(frozen=True)
class Payload:
descriptor: PayloadDescriptor
representation: InlinePayload | ArtifactPayload
A leaf adapter produces a typed transport value:
{
"__type__": "array",
"version": 1,
"library": "numpy",
"metadata": {
"shape": [3, 100, 100],
"dtype": "<f4",
"order": "C",
},
"payload": {
"kind": "inline",
"codec": "none",
"sha256": "...",
"data": b"...",
},
}
Small binary payloads can be inlined. Large payloads are stored through an ArtifactStore and the packet carries an ArtifactRef instead.
MessagePack extension code 42 is reserved for protocol sentinels such as the empty-container fast path.
Wire Envelope
A Wire Envelope carries exchange/API context around a DataContainer. It is distinct from the low-level MessagePack wire representation.
The 0.1.0 implementation provides SSEEnvelope for API/SSE exchanges:
SSEEnvelope
├── agent_id
├── agent_status
├── event_type
├── request_type
├── job_id
├── errors
└── data → DataContainer | None
data may be absent. When present, it represents the data available for that particular exchange event; it does not imply that the data is complete. Streaming updates can therefore carry partial DataContainer values, such as an AudioPacket containing one audio chunk.
The envelope is consumed at the exchange boundary. Agents and Templates continue to receive a DataContainer; they do not need to know whether it arrived locally or through SSE/API transport.
Wire Envelope describes the context of an exchange. Wire Format describes the representation of the data being exchanged.
Example:
from sinapsis_protocol import (
AgentStatus,
EventType,
RequestType,
SSEEnvelope,
serialize_sse_envelope,
)
envelope = SSEEnvelope(
agent_id="agent-1",
agent_status=AgentStatus.RUNNING,
event_type=EventType.DATA,
request_type=RequestType.STREAM,
job_id="job-123",
data=data_container,
)
packet = serialize_sse_envelope(envelope)
Resource limits and safety
Serialization and deserialization support limits including:
maximum packet bytes;
maximum inline bytes;
maximum array bytes;
maximum collection length;
maximum string bytes;
maximum nesting depth.
Binary array decoding validates the expected byte count from shape × dtype.itemsize before constructing the array.
The protocol intentionally does not use automatic pickle/cloudpickle serialization. Arbitrary Python object reconstruction would make the wire format Python-specific and introduce unsafe deserialization behavior.
Versioning policy
Protocol identity:
sinapsis-data / major=1 / minor=0
The major version is the compatibility boundary. Minor releases may add compatible capabilities and optional fields. Individual transport types also have their own versions.
The generic protocol does not promise Python object identity. It promises reconstruction of the supported typed data represented by the wire contract.
Related content
Protocol Quickstart — Python serialization API and examples.
Extending the Protocol — custom packet dataclasses and adapters.
Payload Codecs — formats used inside packet payloads.
Published from Git revision 7b7ba25d627b