Docs /Templates/ sinapsis-llama-index

CodeEmbeddingNodeGenerator

sinapsis-llama-indexEmbeddingsembeddingshuggingfacetextdocuments

Python template · sinapsis_llama_index.templates.code_embedding_node_generator

Overview

Generate embedded code chunks from repository documents.

This template filters loaded documents by programming language and exclusion rules, splits matching files with LlamaIndex's `CodeSplitter`, and stores the embedded `TextNode` objects in `container.generic_data[self.instance_name]`.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-llama-index==1.0.0
uv pip install sinapsis-llama-index==1.0.0
uv add sinapsis-llama-index==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
generic_keys

The key or list of keys for retrieving the document data from the container. If `None`, it will try to use `container.texts`. Defaults to `None`. programming_language (Literal): list of allowed programming language to use with the CodeSplitter

list[str] | None
Optional
Default: null
splitter_args

Configuration for the text splitter (e.g., chunk_lines size, max_chars).

CodeSplitterArgs
Optional
Default: null
language
typing.Literal['python', 'sql', 'javascript', 'java', 'json', 'bash', 'markdown', 'ruby', 'cpp', 'css', 'yaml', 'html']
Optional
Default: "python"
max_chars

The maximum number of characters allowed in a chunk. Defaults to `DEFAULT_MAX_CHARS`.

int
Optional
Default: 1500
chunk_lines

The number of lines to include in each chunk. Defaults to `DEFAULT_CHUNK_LINES`.

int
Optional
Default: 40
include_metadata
bool
Optional
Default: true
chunk_lines_overlap

The number of lines to overlap between consecutive chunks. Defaults to `DEFAULT_LINES_OVERLAP`.

int
Optional
Default: 15
include_prev_next_rel
bool
Optional
Default: true
embedding_configRequired

Configuration for the HuggingFace embedding model (e.g., model name, device).

HFEmbeddingConfig
Required
Default: null
device

The device to run the model on ('cuda', 'cpu', 'auto'). If 'auto', selects 'cuda' if available, otherwise 'cpu'. Defaults to `"auto"`.

typing.Literal['auto', 'cuda', 'cpu']
Optional
Default: "auto"
normalize

Whether to normalize the embeddings. Defaults to `True`.

bool
Optional
Default: true
max_length

Maximum sequence length for the model. Usually inferred if None. Defaults to `None`.

int | None
Optional
Default: null
model_nameRequired

The name or path of the HuggingFace embedding model (e.g., 'BAAI/bge-small-en-v1.5').

str
Required
Default: null
cache_folder

Path to cache directory for downloading/loading models. Defaults to `SINAPSIS_CACHE_DIR`.

str
Optional
Default: "/home/ariel/.cache/sinapsis"
embed_batch_size

Batch size for embedding generation. Defaults to `DEFAULT_EMBED_BATCH_SIZE`.

int
Optional
Default: 10
parallel_process

Whether to use parallel processing if multiple GPUs are available. Defaults to `False`.

bool
Optional
Default: false
text_instruction

Instruction prefix for document embeddings (e.g., for Instructor models). Defaults to `None`.

str | None
Optional
Default: null
query_instruction

Instruction prefix for query embeddings (e.g., for Instructor models). Defaults to `None`.

str | None
Optional
Default: null
trust_remote_code

Allow execution of remote code for custom models. Defaults to `False`.

bool
Optional
Default: false
exclusion_config
FileExclusionConfig
Optional
Default: null
file_path_key

The key in the document's metadata dictionary that contains the full file path string to check against. Defaults to `"file_path"`.

str
Optional
Default: "file_path"
file_type_key

The key in the document's metadata dictionary that contains the file type string. Defaults to `"file_type"`.

str
Optional
Default: "file_type"
endswith_exclude

A list of string suffixes. Documents whose file path (from `file_path_key`) ends with any of these strings will be excluded (e.g., "__init__.py", ".tmp"). Defaults to an empty list.

list[str]
Optional
Default: null
startswith_exclude

A list of string prefixes. Documents whose file path (from `file_path_key`) starts with any of these strings will be excluded. Defaults to an empty list.

list[str]
Optional
Default: null

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: CodeEmbeddingNodeGenerator
  class_name: CodeEmbeddingNodeGenerator
  template_input: InputTemplate
  attributes:
    splitter_args:
      language: python
      chunk_lines: 64
      chunk_lines_overlap: 8
      max_chars: 1024
    embedding_config:
      model_name: Snowflake/snowflake-arctic-embed-xs
      embed_batch_size: 10
      trust_remote_code: true
      device: cpu
    exclusion_config:
      endswith_exclude: ['__init__.py']
      file_path_key: file_path
      file_type_key: file_type
    generic_keys: ['GitLoader-repo']
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "CodeEmbeddingNodeGenerator"
    class_name: "CodeEmbeddingNodeGenerator"
    attributes:
      generic_keys: null
      splitter_args:
        language: "python"
        max_chars: 1500
        chunk_lines: 40
        include_metadata: true
        chunk_lines_overlap: 15
        include_prev_next_rel: true
      embedding_config:
        device: "auto"
        normalize: true
        max_length: null
        model_name: "<required>"
        cache_folder: "/home/ariel/.cache/sinapsis"
        embed_batch_size: 10
        parallel_process: false
        text_instruction: null
        query_instruction: null
        trust_remote_code: false
      exclusion_config:
        file_path_key: "file_path"
        file_type_key: "file_type"
        endswith_exclude: null
        startswith_exclude: null

Inheritance

Immediate parent: sinapsis_llama_index.templates.embedding_node_generator.EmbeddingNodeGenerator

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.