Docs /Templates/ sinapsis-llama-index

LLaMAIndexSemanticCacheLookup

sinapsis-llama-indexLlamaIndexdatabaseembeddingshuggingfacellamaindex

Python template · sinapsis_llama_index.templates.llama_index_semantic_cache_lookup

Overview

Lookup cached LLM responses in PGVector and hydrate conversation packets on hits.

This template performs a semantic lookup using the current user prompt while enforcing exact-match filters for the effective request context via hashed metadata. On a hit, it writes the cached response back into the `LLMConversationPacket` and marks the termination reason as `cache_hit`.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-llama-index==1.0.0
uv pip install sinapsis-llama-index==1.0.0
uv add sinapsis-llama-index==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
db_config

PGVector connection settings.

VectorStoreDBConfig
Optional
Default: null
host

Database host address (hostname or IP). Loaded from PGVECTOR_HOST if not provided. Defaults to `"localhost"`.

str
Optional
Default: "localhost"
port

Database port number. Loaded from PGVECTOR_PORT if not provided. Defaults to `5432`.

int
Optional
Default: 5432
user

Database user name. Loaded from PGVECTOR_USER if not provided. Defaults to `None`.

str | None
Optional
Default: null
db_name

Name of the specific database to connect to. Defaults to `"sinapsis_db"`.

str
Optional
Default: "sinapsis_db"
password

Database password. Loaded from PGVECTOR_PASSWORD if not provided. Defaults to `None`.

str | None
Optional
Default: null
provider

Database provider. Supabase uses the existing database and SSL. Defaults to `"postgres"`.

typing.Literal['postgres', 'supabase']
Optional
Default: "postgres"
table_name

Name of the table within the database containing the vectors. Defaults to `"sinapsis_table"`.

str
Optional
Default: "sinapsis_table"
global_scope

When `True`, ignore `user_id` and `session_id` during cache lookup.

bool
Optional
Default: false
retrieve_args

Base retrieval configuration. Metadata filters are merged with exact-match cache filters and combined with `AND`.

RetrieveArgs
Optional
Default: null
doc_ids

Optional document identifiers to restrict retrieval to. Defaults to `None`.

list[str] | None
Optional
Default: null
node_ids

Optional node identifiers to restrict retrieval to. Defaults to `None`.

list[str] | None
Optional
Default: null
threshold

The minimum similarity score required for a retrieved node to be considered relevant and included in the final results. Defaults to `0.85`.

float
Optional
Default: 0.85
query_mode

The LlamaIndex query mode to use for retrieval. Defaults to `"default"`.

VectorStoreQueryMode
Optional
Default: "default"
filter_condition

Logical condition used to combine `metadata_filters`. Defaults to `"and"`.

typing.Literal['and', 'or', 'not']
Optional
Default: "and"
metadata_filters

Optional metadata filters applied before vector results are returned. Defaults to an empty list.

list[sinapsis_llama_index.helpers.schemas.MetadataFilterConfig]
Optional
Default: null
similarity_top_k

The maximum number of potentially relevant nodes to retrieve from the vector store before applying the similarity threshold. Defaults to `1`.

int
Optional
Default: 1
metadata_filters[]
object
Optional
Default: Not specified
keyRequired

Metadata key to filter on.

str
Required
Default: null
valueRequired

Metadata value used by the filter.

int | float | str | list[int] | list[float] | list[str] | None
Required
Default: null
operator

Comparison operator applied to the metadata field. Defaults to `FilterOperator.EQ`.

FilterOperator
Optional
Default: "=="
embedding_configRequired

Embedding model used for prompt similarity search.

HFEmbeddingConfig
Required
Default: null
device

The device to run the model on ('cuda', 'cpu', 'auto'). If 'auto', selects 'cuda' if available, otherwise 'cpu'. Defaults to `"auto"`.

typing.Literal['auto', 'cuda', 'cpu']
Optional
Default: "auto"
normalize

Whether to normalize the embeddings. Defaults to `True`.

bool
Optional
Default: true
max_length

Maximum sequence length for the model. Usually inferred if None. Defaults to `None`.

int | None
Optional
Default: null
model_nameRequired

The name or path of the HuggingFace embedding model (e.g., 'BAAI/bge-small-en-v1.5').

str
Required
Default: null
cache_folder

Path to cache directory for downloading/loading models. Defaults to `SINAPSIS_CACHE_DIR`.

str
Optional
Default: "/home/ariel/.cache/sinapsis"
embed_batch_size

Batch size for embedding generation. Defaults to `DEFAULT_EMBED_BATCH_SIZE`.

int
Optional
Default: 10
parallel_process

Whether to use parallel processing if multiple GPUs are available. Defaults to `False`.

bool
Optional
Default: false
text_instruction

Instruction prefix for document embeddings (e.g., for Instructor models). Defaults to `None`.

str | None
Optional
Default: null
query_instruction

Instruction prefix for query embeddings (e.g., for Instructor models). Defaults to `None`.

str | None
Optional
Default: null
trust_remote_code

Allow execution of remote code for custom models. Defaults to `False`.

bool
Optional
Default: false

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: LLaMAIndexSemanticCacheLookup
  class_name: LLaMAIndexSemanticCacheLookup
  template_input: InputTemplate
  attributes:
    embedding_config:
      model_name: nomic-ai/nomic-embed-text-v1.5
      trust_remote_code: true
      device: cpu
    db_config:
      user: my_user
      password: my_password
      db_name: my_db
      table_name: semantic_cache
    retrieve_args:
      similarity_top_k: 1
      threshold: 0.92
      metadata_filters:
        - key: model_name
          value: Qwen/Qwen3-1.7B
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "LLaMAIndexSemanticCacheLookup"
    class_name: "LLaMAIndexSemanticCacheLookup"
    attributes:
      db_config:
        host: "localhost"
        port: 5432
        user: null
        db_name: "sinapsis_db"
        password: null
        provider: "postgres"
        table_name: "sinapsis_table"
      global_scope: false
      retrieve_args:
        doc_ids: null
        node_ids: null
        threshold: 0.85
        query_mode: {}
        filter_condition: "and"
        metadata_filters: null
        similarity_top_k: 1
        metadata_filters[]:
          key: "<required>"
          value: "<required>"
          operator: {}
      embedding_config:
        device: "auto"
        normalize: true
        max_length: null
        model_name: "<required>"
        cache_folder: "/home/ariel/.cache/sinapsis"
        embed_batch_size: 10
        parallel_process: false
        text_instruction: null
        query_instruction: null
        trust_remote_code: false

Inheritance

Immediate parent: sinapsis_core.template_base.template.Template

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.