CodeEmbeddingNodeGenerator
Python template · sinapsis_llama_index.templates.code_embedding_node_generator
Overview
Generate embedded code chunks from repository documents.
This template filters loaded documents by programming language and exclusion rules, splits matching files with LlamaIndex's `CodeSplitter`, and stores the embedded `TextNode` objects in `container.generic_data[self.instance_name]`.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis-llama-index==1.0.0uv pip install sinapsis-llama-index==1.0.0uv add sinapsis-llama-index==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: null
The key or list of keys for retrieving the document data from the container. If `None`, it will try to use `container.texts`. Defaults to `None`. programming_language (Literal): list of allowed programming language to use with the CodeSplitter
Optional
Default: null
Configuration for the text splitter (e.g., chunk_lines size, max_chars).
Optional
Default: null
Optional
Default: "python"
The maximum number of characters allowed in a chunk. Defaults to `DEFAULT_MAX_CHARS`.
Optional
Default: 1500
The number of lines to include in each chunk. Defaults to `DEFAULT_CHUNK_LINES`.
Optional
Default: 40
Optional
Default: true
The number of lines to overlap between consecutive chunks. Defaults to `DEFAULT_LINES_OVERLAP`.
Optional
Default: 15
Optional
Default: true
Configuration for the HuggingFace embedding model (e.g., model name, device).
Required
Default: null
The device to run the model on ('cuda', 'cpu', 'auto'). If 'auto', selects 'cuda' if available, otherwise 'cpu'. Defaults to `"auto"`.
Optional
Default: "auto"
Whether to normalize the embeddings. Defaults to `True`.
Optional
Default: true
Maximum sequence length for the model. Usually inferred if None. Defaults to `None`.
Optional
Default: null
The name or path of the HuggingFace embedding model (e.g., 'BAAI/bge-small-en-v1.5').
Required
Default: null
Path to cache directory for downloading/loading models. Defaults to `SINAPSIS_CACHE_DIR`.
Optional
Default: "/home/ariel/.cache/sinapsis"
Batch size for embedding generation. Defaults to `DEFAULT_EMBED_BATCH_SIZE`.
Optional
Default: 10
Whether to use parallel processing if multiple GPUs are available. Defaults to `False`.
Optional
Default: false
Instruction prefix for document embeddings (e.g., for Instructor models). Defaults to `None`.
Optional
Default: null
Instruction prefix for query embeddings (e.g., for Instructor models). Defaults to `None`.
Optional
Default: null
Allow execution of remote code for custom models. Defaults to `False`.
Optional
Default: false
Optional
Default: null
The key in the document's metadata dictionary that contains the full file path string to check against. Defaults to `"file_path"`.
Optional
Default: "file_path"
The key in the document's metadata dictionary that contains the file type string. Defaults to `"file_type"`.
Optional
Default: "file_type"
A list of string suffixes. Documents whose file path (from `file_path_key`) ends with any of these strings will be excluded (e.g., "__init__.py", ".tmp"). Defaults to an empty list.
Optional
Default: null
A list of string prefixes. Documents whose file path (from `file_path_key`) starts with any of these strings will be excluded. Defaults to an empty list.
Optional
Default: null
Configuration example
Example from the Python docstring:
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: CodeEmbeddingNodeGenerator
class_name: CodeEmbeddingNodeGenerator
template_input: InputTemplate
attributes:
splitter_args:
language: python
chunk_lines: 64
chunk_lines_overlap: 8
max_chars: 1024
embedding_config:
model_name: Snowflake/snowflake-arctic-embed-xs
embed_batch_size: 10
trust_remote_code: true
device: cpu
exclusion_config:
endswith_exclude: ['__init__.py']
file_path_key: file_path
file_type_key: file_type
generic_keys: ['GitLoader-repo']Show schema-based configuration skeleton
templates:
- template_name: "CodeEmbeddingNodeGenerator"
class_name: "CodeEmbeddingNodeGenerator"
attributes:
generic_keys: null
splitter_args:
language: "python"
max_chars: 1500
chunk_lines: 40
include_metadata: true
chunk_lines_overlap: 15
include_prev_next_rel: true
embedding_config:
device: "auto"
normalize: true
max_length: null
model_name: "<required>"
cache_folder: "/home/ariel/.cache/sinapsis"
embed_batch_size: 10
parallel_process: false
text_instruction: null
query_instruction: null
trust_remote_code: false
exclusion_config:
file_path_key: "file_path"
file_type_key: "file_type"
endswith_exclude: null
startswith_exclude: null
Inheritance
sinapsis_llama_index.templates.embedding_node_generator.EmbeddingNodeGeneratorEnvironment variables
Environment definitions belong to their canonical Python modules.
No directly related environment variables listed.