Docs /guides/ Extending Sinapsis Protocol
guides / protocol

Extending Sinapsis Protocol

Register custom dataclasses and leaf payload adapters with stable wire identities.

protocoladapterspythonDocumentation latest

Use a registered dataclass for structured recursive packet types; use a leaf/object adapter for a custom binary representation of one leaf value. Both extend the same DataContainer protocol and neither introduces a competing agent-level transport.

For installation and first serialization, begin with Protocol Quickstart. For the wire contract, see Wire Format.

Adding a new packet type

For a new structured object such as MyCustomPacket, use a dataclass and register it with a stable wire name.

python
from sinapsis.data_transport import dataclass, field

from sinapsis_protocol import default_registry, serialize_data_container


@dataclass
class MyCustomPacket:
    content: bytes
    label: str = ""
    metadata: dict[str, str] = field(default_factory=dict)


registry = default_registry()
registry.register_dataclass(
    MyCustomPacket,
    type_name="myapp:my-custom-packet",
    version=1,
)

The dataclass itself is then traversed recursively. If it is contained inside a DataContainer field, pass the registry to the normal serializer:

python
packet = serialize_data_container(
    data_container,
    registry=registry,
)

Stable names matter

Prefer a protocol identity such as:

text
myapp:my-custom-packet

over a Python implementation path such as:

text
my_package.models.MyCustomPacket

The wire name is part of the protocol contract. Python modules and class locations can change without changing the wire identity.

Versioning a custom packet

If the schema changes incompatibly, register a new version rather than silently changing the meaning of version 1:

python
registry.register_dataclass(
    MyCustomPacketV2,
    type_name="myapp:my-custom-packet",
    version=2,
)

A future protocol implementation can then provide explicit migration logic between versions rather than guessing how an old payload should be interpreted.

Adding a new leaf/object type with an adapter

Use an adapter when the object is fundamentally a leaf payload rather than a recursive dataclass tree.

For example, suppose an application has a custom MyEmbedding object that should be represented by a compact binary payload:

python
from sinapsis_protocol import default_registry
from sinapsis_protocol.wire.constants import PayloadCodec, PayloadKind
from sinapsis_protocol.wire.models import (
    InlinePayload,
    Payload,
    PayloadDescriptor,
)


class MyEmbeddingAdapter:
    type_name = "embedding"
    library = "myapp"
    version = 1

    @classmethod
    def can_encode(cls, value: object) -> bool:
        return isinstance(value, MyEmbedding)

    @classmethod
    def encode(cls, value: object, context):
        data = value.to_bytes()
        return Payload(
            descriptor=PayloadDescriptor(
                kind=PayloadKind.BYTES,
                codec=PayloadCodec.NONE,
                version=cls.version,
            ),
            representation=InlinePayload(data=data),
        )

    @classmethod
    def decode(cls, value, context):
        data = value.representation.data
        return MyEmbedding.from_bytes(data)


registry = default_registry()
registry.register(MyEmbeddingAdapter)

Then use that registry with the normal recursive serializer:

python
packet = serialize_data_container(data_container, registry=registry)
restored = deserialize_data_container(packet, registry=registry)

The important distinction is:

  • dataclass registration = recursively serialize the object's fields;

  • leaf adapter = own the complete representation of that object as a transport value.

Explicit adapters take precedence over the built-in lazy adapter specifications, so an application can override a built-in representation when necessary.

Protobuf as a payload codec

Protobuf is useful when a particular payload has a stable, compact, language-neutral schema.

Within UDT, protobuf is a payload codec, not a second top-level transport protocol.

Conceptually:

text
DataContainer
└── MyProtoPacket
    └── protobuf message
         ↓
    protobuf serialization
         ↓
    payload bytes
         ↓
    MessagePack DataContainer wire representation

This preserves the universal agent boundary while allowing a specialized representation for one payload.

The same pattern applies to other payload formats:

text
NumPy      → raw binary
Torch      → raw binary
DataFrame  → Arrow IPC
File       → ArtifactRef
bytes      → binary
Protobuf   → protobuf bytes
Custom     → registered payload codec

The rule is:

Specialize the payload, not the universal container boundary.

Registry and extension model

The registry has two extension mechanisms:

text
AdapterRegistry
├── DataclassRegistry
│   └── structured recursive packet types
│
└── Leaf adapters
    ├── NumPy
    ├── Torch
    ├── Pandas
    ├── Polars
    └── application overrides

Built-in optional adapters are loaded lazily. Applications can explicitly register an adapter to override a built-in representation.

Unknown Python objects are not automatically pickled. If a value is not a supported primitive, dataclass, or registered adapter, serialization fails with UnsupportedTypeError.

Choosing the right payload representation

See Payload Codecs for Protobuf, Arrow, raw binary, and codec trade-offs. Unknown objects are not implicitly serialized with pickle.

Published from Git revision 7b7ba25d627b