Docs /architecture/ Template Model
architecture / models

Template Model

Template input/output contracts, enrichment and DataSplit/DataMerge semantics.

templatesdataflowarchitectureDocumentation latest

This is the detailed Template contract. For a concise introduction, read Templates. The larger graph specification is Agent Model.

1. Purpose

A Template is the fundamental transformation unit in the Sinapsis execution model.

A Template consumes a DataContainer, performs a defined operation, and produces a DataContainer.

The core contract is:

text
Template:
    DataContainer → DataContainer

The Template Model defines the semantics of this transformation.

It does not define serialization. Data representation and wire exchange are defined by Universal Data Transport (UDT).


2. The DataContainer Contract

A normal Template has exactly:

  • one DataContainer input

  • one DataContainer output

Conceptually:

text
DataContainer
      │
      ▼
   Template
      │
      ▼
DataContainer

This rule gives Templates a uniform interface regardless of what data they process.

A Template may operate on:

  • images

  • audio

  • text

  • files

  • tensors

  • dataframes

  • annotations

  • metadata

  • custom packets

  • combinations of these

The Template does not need a separate input/output type for every possible payload.


3. Template Operations

A Template may:

  • consume existing packets;

  • transform packet contents;

  • add packets;

  • remove packets;

  • replace packets;

  • modify metadata;

  • add annotations;

  • enrich existing packets;

  • derive new data from existing packets;

  • create an empty DataContainer when appropriate.

For example, an object-detection Template may receive:

text
DataContainer
    └── ImagePacket
          └── image

and produce:

text
DataContainer
    └── ImagePacket
          ├── image
          └── annotations
                ├── labels
                ├── bounding boxes
                └── masks

A downstream Template can then consume those annotations.

For example:

text
Image
  │
  ▼
Object Detection
  │
  ├── labels
  ├── bounding boxes
  └── masks
  │
  ▼
Crop / Draw / Embed

The output remains a DataContainer throughout.


4. Templates Are Graph Nodes

Templates are intended to be composed into graphs.

A Template therefore represents a node with explicit data-flow boundaries rather than an isolated function with an application-specific interface.

A simple graph is:

text
DataContainer
      │
      ▼
 Template A
      │
      ▼
 Template B
      │
      ▼
 Template C
      │
      ▼
DataContainer

The graph determines how container data flows between Templates.


5. Declarative Graph Semantics

Template graphs are declarative.

The graph describes:

  • which Templates exist;

  • which Templates are connected;

  • how data flows between them;

  • where branching occurs;

  • where branches merge;

  • what dependencies exist between nodes.

The graph is not merely an imperative sequence of Python calls.

A runtime may choose how to execute the graph, provided it preserves the declared data-flow semantics.

A typical graph is a DAG:

text
A → B → C

but the conceptual Template Model does not need to prohibit future graph structures solely because the common implementation uses DAGs.


6. Branching

Branching is explicit.

A Template graph may need to send the same logical input to multiple downstream Templates:

◇ DiagramDrag nodes · Scroll to zoom · Drag canvas to pan
flowchart TD
    INPUT[DataContainer] --> SPLIT[DataSplit]
    SPLIT --> A[Template A]
    SPLIT --> B[Template B]
    A --> DA[DataContainer A]
    B --> DB[DataContainer B]

Green-bordered nodes reference Sinapsis templates. Hover or focus them to fetch documentation.

View Mermaid source
mermaid
flowchart TD
    INPUT[DataContainer] --> SPLIT[DataSplit]
    SPLIT --> A[Template A]
    SPLIT --> B[Template B]
    A --> DA[DataContainer A]
    B --> DB[DataContainer B]

The branching operation is represented explicitly rather than being an implicit side effect of multiple references to the same Python object.

This makes ownership and mutation semantics visible in the graph.


7. DataSplit

DataSplit is a special Template used to establish controlled fan-out.

It is an explicit exception to the normal one-input/one-output Template rule because its purpose is to create multiple downstream data paths.

Conceptually:

◇ DiagramDrag nodes · Scroll to zoom · Drag canvas to pan
flowchart TD
    DC[DataContainer] --> S[DataSplit]
    S --> DCA[DataContainer A]
    S --> DCB[DataContainer B]

Green-bordered nodes reference Sinapsis templates. Hover or focus them to fetch documentation.

View Mermaid source
mermaid
flowchart TD
    DC[DataContainer] --> S[DataSplit]
    S --> DCA[DataContainer A]
    S --> DCB[DataContainer B]

DataSplit may operate in:

  • shallow copy mode

  • deep copy mode

The selected mode determines what downstream Templates receive.

Shallow copy

A shallow copy creates a new container structure while retaining references to nested objects where the copy semantics permit it.

This can be useful when data is treated as immutable or when copying large payloads would be unnecessarily expensive.

Deep copy

A deep copy recursively copies the contained mutable object graph.

This provides stronger isolation between downstream branches at the cost of additional memory and processing.

The graph should make the chosen semantics explicit.


8. Why DataSplit Exists

Python objects are passed by reference.

Without explicit branching semantics, two downstream Templates could accidentally receive references to the same mutable object.

For example:

text
             same object
              /                   ▼        ▼
        Template A  Template B
             │        │
             └── mutate ──┘

One Template could unintentionally modify data observed by the other.

DataSplit establishes a controlled ownership boundary before data is presented to downstream Templates.

The purpose is not merely to copy data. It is to make branch semantics explicit and predictable.


9. DataMerge

DataMerge is the corresponding explicit fan-in operation.

It combines multiple branch outputs into a downstream data flow.

Conceptually:

◇ DiagramDrag nodes · Scroll to zoom · Drag canvas to pan
flowchart LR
    A[Template A] --> M[DataMerge]
    B[Template B] --> M
    M --> C[Template C]

Green-bordered nodes reference Sinapsis templates. Hover or focus them to fetch documentation.

View Mermaid source
mermaid
flowchart LR
    A[Template A] --> M[DataMerge]
    B[Template B] --> M
    M --> C[Template C]

DataMerge is also an explicit exception to the normal one-input/one-output Template rule because it consumes multiple upstream branches.

The merge semantics should define how packets, metadata, identifiers, and conflicting fields are combined.


10. Empty DataContainers

A Template may produce an empty DataContainer.

This is useful when:

  • no output data was produced;

  • a filtering operation removes all packets;

  • a downstream stage should receive an explicit empty result;

  • a computation intentionally produces no data.

An empty container remains a valid value in the Template contract.


11. Mutation and Enrichment

Templates may enrich existing packets instead of replacing them.

For example:

text
ImagePacket
    ├── content
    └── annotations

An object detector may append annotations:

text
ImagePacket
    ├── content
    └── annotations
          ├── Detection(label="cat", bbox=...)
          ├── Detection(label="dog", bbox=...)
          └── ...

A downstream Template can consume the annotations to:

  • draw detections;

  • crop detected objects;

  • calculate embeddings;

  • filter detections;

  • create derived packets.

This supports incremental enrichment of a DataContainer as it passes through a graph.


12. Template Independence from Payload Representation

A Template operates on the semantic DataContainer model.

It should not need to know whether a payload was transported using:

  • MessagePack;

  • Arrow;

  • protobuf;

  • raw binary;

  • an artifact reference;

  • another UDT payload codec.

Those concerns belong to UDT.

The separation is:

text
Template Model
    │
    │ semantic data transformation
    ▼
DataContainer
    │
    │ representation/exchange
    ▼
UDT

13. Relationship to UDT

UDT defines the representation and exchange of the data:

How is a DataContainer represented and transported?

The Template Model defines computation over that data:

How does a Template consume and produce a DataContainer?

Therefore:

text
Template
    ↓
DataContainer
    ↓
UDT representation
    ↓
Wire Format

A Template does not become a different kind of component merely because its input or output is transported remotely.


14. Core Rules

The Template Model can be summarized by these rules:

  1. A normal Template receives exactly one DataContainer.

  2. A normal Template produces exactly one DataContainer.

  3. Templates may transform, enrich, add, remove, or replace data.

  4. Template graphs are declarative.

  5. Branching is explicit.

  6. DataSplit establishes branch/copy semantics.

  7. DataMerge establishes explicit fan-in semantics.

  8. Shallow and deep split modes are distinct and intentional.

  9. Templates operate on semantic DataContainer objects, not wire-format details.

  10. UDT defines representation and exchange; the Template Model defines transformation semantics.

Published from Git revision 7b7ba25d627b