Protocol Implementation
Python sinapsis-protocol 0.1.0 architecture, modules and scope.
Sinapsis Protocol Implementation
sinapsis-protocol is the Python implementation of the UDT Architecture. These notes describe the source document's 0.1.0 design, not a verification against a live release.
Architecture
flowchart TD
A[Sinapsis DataContainer] --> B[Recursive dataclass walker]
B --> C[Nested DataPackets / dataclasses]
C --> D{Payload}
D -->|NumPy| E[NumPy payload codec]
D -->|Torch| F[Torch payload codec]
D -->|Pandas / Polars| G[Arrow payload codec]
D -->|bytes / string / path| H[Core payload representation]
D -->|Protobuf / custom type| I[Registered payload codec]
E --> J[Encoded payload]
F --> J
G --> J
H --> J
I --> J
J --> K[DataContainer wire representation]
K --> L{Payload size}
L -->|small| M[Inline payload]
L -->|large| N[ArtifactRef]
M --> O[Versioned MessagePack wire format]
N --> O
O --> P[Wire representation]
P --> Q[Transport]
Q --> R[Deserializer]
R --> S[Trusted dataclass / payload codec registry]
S --> T[Reconstructed DataContainer]
Green-bordered nodes reference Sinapsis templates. Hover or focus them to fetch documentation.
View Mermaid source
flowchart TD
A[Sinapsis DataContainer] --> B[Recursive dataclass walker]
B --> C[Nested DataPackets / dataclasses]
C --> D{Payload}
D -->|NumPy| E[NumPy payload codec]
D -->|Torch| F[Torch payload codec]
D -->|Pandas / Polars| G[Arrow payload codec]
D -->|bytes / string / path| H[Core payload representation]
D -->|Protobuf / custom type| I[Registered payload codec]
E --> J[Encoded payload]
F --> J
G --> J
H --> J
I --> J
J --> K[DataContainer wire representation]
K --> L{Payload size}
L -->|small| M[Inline payload]
L -->|large| N[ArtifactRef]
M --> O[Versioned MessagePack wire format]
N --> O
O --> P[Wire representation]
P --> Q[Transport]
Q --> R[Deserializer]
R --> S[Trusted dataclass / payload codec registry]
S --> T[Reconstructed DataContainer]
Note: Protobuf and other specialized formats are payload codecs. They encode individual payloads inside DataPackets; they do not bypass the UDT container or wire format.
The generic path is:
DataContainer
↓
recursive dataclass walker
↓
nested packet dataclasses
↓
leaf dispatch
↓
typed payload representation / inline bytes / ArtifactRef
↓
versioned MessagePack envelope
A specialized application can intentionally bypass this path:
Application contract
↓
protobuf schema
↓
protobuf bytes
That is useful when two endpoints have a fixed, tightly controlled contract and do not need the generic DataContainer protocol.
Project layout
src/sinapsis_protocol/
├── __init__.py
├── container/
│ ├── serialize.py # DataContainer integration
│ └── deserialize.py
├── wire/
│ ├── constants.py # protocol vocabulary / enum-like values
│ ├── models.py # strongly typed wire and payload models
│ ├── format.py # MessagePack wire format and versioning
│ └── sentinels.py # protocol-level sentinel states
├── dataclass/
│ ├── registry.py # structured packet type registry
│ ├── encode.py # recursive dataclass encoding
│ └── decode.py # recursive dataclass decoding
├── codecs/
│ ├── base.py # payload codec contract
│ ├── registry.py # lazy codec registry and overrides
│ ├── numpy.py
│ ├── torch.py
│ ├── pandas.py
│ ├── polars.py
│ └── cv2.py
├── artifacts/
│ ├── base.py # artifact storage contract
│ ├── ref.py # ArtifactRef
│ └── local.py # local artifact store
├── types.py # shared protocol contracts
└── errors.py # protocol-specific exceptions
What 0.1.0 does and does not promise
Included
recursive
DataContainer/dataclass serialization;MessagePack framing;
protocol and transport-type versions;
strongly typed internal protocol models;
built-in NumPy, Torch, Pandas, and Polars adapters;
inline and artifact-backed binary payloads;
configurable resource limits;
explicit custom dataclass registration;
explicit leaf-adapter overrides;
no implicit pickle fallback.
Deliberately outside the generic protocol
arbitrary Python object graphs;
live process state;
automatic serialization of arbitrary
torch.nn.Moduleinstances;implicit pickle/cloudpickle execution;
a separate top-level protocol for each payload type;
application-specific fixed schemas replacing
DataContainer.
Protobuf or another fixed schema can still be used inside a DataPacket when that payload benefits from it.
Development
pip install -e '.[dev]'pytestThe test suite should cover both protocol mechanics and real sinapsis_core integration before treating a release as production-ready.
Licensing
The original implementation document points to its project's LICENSE file. That file was not among the six supplied documents, so no license file or license terms are reproduced in this documentation repository. Check the implementation repository before reusing code outside its existing license.
Further reading
Published from Git revision 7b7ba25d627b