Docs /Templates/ sinapsis-huggingface-transformers

PaliGemmaDetection

sinapsis-huggingface-transformersHuggingFacehuggingfaceimagepaligemmamodels

Python template · sinapsis_huggingface_transformers.templates.pali_gemma.pali_gemma_detection

Overview

Implementation of PaliGemma object detection pipeline.

The template inherits functionality from its base class, extending the functionality to run inference on an image and to identify the objects from the attributes.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-huggingface-transformers==1.0.0
uv pip install sinapsis-huggingface-transformers==1.0.0
uv add sinapsis-huggingface-transformers==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

device
typing.Literal['cuda', 'cpu']
Optional
Default: "cpu"
prompt
str
Optional
Default: "<image>caption en"
metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
model_path
typing.Literal['google/paligemma2-3b-pt-224']
Optional
Default: "google/paligemma2-3b-pt-224"
torch_dtype
typing.Literal['float16', 'float32']
Optional
Default: "float16"
max_new_tokens
int
Optional
Default: 200
processor_pathRequired
str
Required
Default: null
model_cache_dir
str
Optional
Default: "/home/ariel/.cache/sinapsis"
objects_to_detectRequired

Target objects to detect, can be a single string or list of strings

str | list[str]
Required
Default: null

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: PaliGemmaDetection
  class_name: PaliGemmaDetection
  template_input: InputTemplate
  attributes:
    model_path: '/path/to/paligemma/model'
    processor_path: '`/path/to/processor'
    model_cache_dir: /path/to/cache/dir
    device: 'cuda'
    max_new_tokens: 200
    torch_dtype: float16
    prompt: <image> caption en
    objects_to_detect: 'object to detect'
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "PaliGemmaDetection"
    class_name: "PaliGemmaDetection"
    attributes:
      device: "cpu"
      prompt: "<image>caption en"
      model_path: "google/paligemma2-3b-pt-224"
      torch_dtype: "float16"
      max_new_tokens: 200
      processor_path: "<required>"
      model_cache_dir: "/home/ariel/.cache/sinapsis"
      objects_to_detect: "<required>"

Inheritance

Immediate parent: sinapsis_huggingface_transformers.templates.pali_gemma.pali_gemma_inference.PaliGemmaInference

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.