Docs /Templates/ sinapsis-llama-index

EmbeddingNodeGenerator

sinapsis-llama-indexEmbeddingsembeddingshuggingfacetextdocuments

Python template · sinapsis_llama_index.templates.embedding_node_generator

Overview

Generate embedded `TextNode` objects from text documents.

This template loads documents either from `container.texts` or from generic-data keys, splits them with a LlamaIndex text splitter, and attaches HuggingFace embeddings to the resulting nodes. The nodes are then stored in `container.generic_data[self.instance_name]` for later insertion into PGVector.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-llama-index==1.0.0
uv pip install sinapsis-llama-index==1.0.0
uv add sinapsis-llama-index==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
generic_keys

The list of keys for retrieving the document data from the container. If `None`, it will try to use `container.texts`. Defaults to `None`.

list[str] | None
Optional
Default: null
splitter_args

Configuration for the text splitter (e.g., chunk size, overlap).

SentenceSplitterArgs
Optional
Default: null
separator

The separator string to use when joining text chunks. Defaults to `" "`.

str
Optional
Default: " "
chunk_size

The maximum size (in tokens or characters, depending on the splitter) of each chunk. Defaults to `DEFAULT_CHUNK_SIZE`.

int
Optional
Default: 1024
chunk_overlap

The number of tokens/characters to overlap between consecutive chunks. Defaults to `SENTENCE_CHUNK_OVERLAP`.

int
Optional
Default: 200
include_metadata
bool
Optional
Default: true
paragraph_separator

The string used to identify paragraph breaks within the text. Defaults to `DEFAULT_PARAGRAPH_SEP`.

str
Optional
Default: "\n\n\n"
include_prev_next_rel
bool
Optional
Default: true
secondary_chunking_regex

A regex pattern used for secondary splitting within larger chunks. Defaults to `CHUNKING_REGEX`.

str
Optional
Default: "[^,.;。?!]+[,.;。?!]?|[,.;。?!]"
embedding_configRequired

Configuration for the HuggingFace embedding model (e.g., model name, device).

HFEmbeddingConfig
Required
Default: null
device

The device to run the model on ('cuda', 'cpu', 'auto'). If 'auto', selects 'cuda' if available, otherwise 'cpu'. Defaults to `"auto"`.

typing.Literal['auto', 'cuda', 'cpu']
Optional
Default: "auto"
normalize

Whether to normalize the embeddings. Defaults to `True`.

bool
Optional
Default: true
max_length

Maximum sequence length for the model. Usually inferred if None. Defaults to `None`.

int | None
Optional
Default: null
model_nameRequired

The name or path of the HuggingFace embedding model (e.g., 'BAAI/bge-small-en-v1.5').

str
Required
Default: null
cache_folder

Path to cache directory for downloading/loading models. Defaults to `SINAPSIS_CACHE_DIR`.

str
Optional
Default: "/home/ariel/.cache/sinapsis"
embed_batch_size

Batch size for embedding generation. Defaults to `DEFAULT_EMBED_BATCH_SIZE`.

int
Optional
Default: 10
parallel_process

Whether to use parallel processing if multiple GPUs are available. Defaults to `False`.

bool
Optional
Default: false
text_instruction

Instruction prefix for document embeddings (e.g., for Instructor models). Defaults to `None`.

str | None
Optional
Default: null
query_instruction

Instruction prefix for query embeddings (e.g., for Instructor models). Defaults to `None`.

str | None
Optional
Default: null
trust_remote_code

Allow execution of remote code for custom models. Defaults to `False`.

bool
Optional
Default: false

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: EmbeddingNodeGenerator
  class_name: EmbeddingNodeGenerator
  template_input: InputTemplate
  attributes:
    splitter_args:
      chunk_size: 256
      chunk_overlap: 32
      separator: " "
    embedding_config:
      model_name: Snowflake/snowflake-arctic-embed-xs
      normalize: true
      embed_batch_size: 10
      trust_remote_code: true
      device: auto
      parallel_process: false
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "EmbeddingNodeGenerator"
    class_name: "EmbeddingNodeGenerator"
    attributes:
      generic_keys: null
      splitter_args:
        separator: " "
        chunk_size: 1024
        chunk_overlap: 200
        include_metadata: true
        paragraph_separator: "\n\n\n"
        include_prev_next_rel: true
        secondary_chunking_regex: "[^,.;。?!]+[,.;。?!]?|[,.;。?!]"
      embedding_config:
        device: "auto"
        normalize: true
        max_length: null
        model_name: "<required>"
        cache_folder: "/home/ariel/.cache/sinapsis"
        embed_batch_size: 10
        parallel_process: false
        text_instruction: null
        query_instruction: null
        trust_remote_code: false

Inheritance

Immediate parent: sinapsis_core.template_base.template.Template

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.