PaliGemmaDetection
Python template · sinapsis_huggingface_transformers.templates.pali_gemma.pali_gemma_detection
Overview
Implementation of PaliGemma object detection pipeline.
The template inherits functionality from its base class, extending the functionality to run inference on an image and to identify the objects from the attributes.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
Terminalbash
pip install sinapsis-huggingface-transformers==1.0.0uv pip install sinapsis-huggingface-transformers==1.0.0uv add sinapsis-huggingface-transformers==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
device
typing.Literal['cuda', 'cpu']
Optional
Default: "cpu"
Optional
Default: "cpu"
prompt
str
Optional
Default: "<image>caption en"
Optional
Default: "<image>caption en"
metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
Optional
Default: null
model_path
typing.Literal['google/paligemma2-3b-pt-224']
Optional
Default: "google/paligemma2-3b-pt-224"
Optional
Default: "google/paligemma2-3b-pt-224"
torch_dtype
typing.Literal['float16', 'float32']
Optional
Default: "float16"
Optional
Default: "float16"
max_new_tokens
int
Optional
Default: 200
Optional
Default: 200
processor_pathRequired
str
Required
Default: null
Required
Default: null
model_cache_dir
str
Optional
Default: "/home/ariel/.cache/sinapsis"
Optional
Default: "/home/ariel/.cache/sinapsis"
objects_to_detectRequired
Target objects to detect, can be a single string or list of strings
str | list[str]
Required
Default: null
Required
Default: null
Configuration example
Example from the Python docstring:
yaml
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: PaliGemmaDetection
class_name: PaliGemmaDetection
template_input: InputTemplate
attributes:
model_path: '/path/to/paligemma/model'
processor_path: '`/path/to/processor'
model_cache_dir: /path/to/cache/dir
device: 'cuda'
max_new_tokens: 200
torch_dtype: float16
prompt: <image> caption en
objects_to_detect: 'object to detect'Show schema-based configuration skeleton
yaml
templates:
- template_name: "PaliGemmaDetection"
class_name: "PaliGemmaDetection"
attributes:
device: "cpu"
prompt: "<image>caption en"
model_path: "google/paligemma2-3b-pt-224"
torch_dtype: "float16"
max_new_tokens: 200
processor_path: "<required>"
model_cache_dir: "/home/ariel/.cache/sinapsis"
objects_to_detect: "<required>"
Inheritance
Immediate parent:
sinapsis_huggingface_transformers.templates.pali_gemma.pali_gemma_inference.PaliGemmaInferenceEnvironment variables
Environment definitions belong to their canonical Python modules.
No directly related environment variables listed.