PaliGemmaInference
Python template · sinapsis_huggingface_transformers.templates.pali_gemma.pali_gemma_inference
Overview
Implementation of PaliGemma inference pipeline for image processing and caption generation.
This class handles the inference process for PaliGemma models, including image processing, caption generation, and annotation creation. It supports both basic captioning and detection/segmentation tasks.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis-huggingface-transformers==1.0.0uv pip install sinapsis-huggingface-transformers==1.0.0uv add sinapsis-huggingface-transformers==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: "cpu"
Prompt to run the inference (default: "<image>caption en") The <image> token is essential as it serves as a marker that tells the model where to look at the image when processing the input. This token enables the model… Read full field description
Prompt to run the inference (default: "<image>caption en") The <image> token is essential as it serves as a marker that tells the model where to look at the image when processing the input. This token enables the model to understand the relationship between the visual and textual components during processing. Example prompts: - "<image>caption en" -> Generates a basic caption in English - "<image>What objects can you see in this image?" -> Lists objects in the image
Optional
Default: "<image>caption en"
Optional
Default: null
Optional
Default: "google/paligemma2-3b-pt-224"
Optional
Default: "float16"
Optional
Default: 200
Required
Default: null
Optional
Default: "/home/ariel/.cache/sinapsis"
Configuration example
Example from the Python docstring:
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: PaliGemmaInference
class_name: PaliGemmaInference
template_input: InputTemplate
attributes:
model_path: '/path/to/paligemma/model'
processor_path: '`/path/to/processor'
model_cache_dir: /path/to/cache/dir
device: 'cuda'
max_new_tokens: 200
torch_dtype: float16
prompt: <image> caption enShow schema-based configuration skeleton
templates:
- template_name: "PaliGemmaInference"
class_name: "PaliGemmaInference"
attributes:
device: "cpu"
prompt: "<image>caption en"
model_path: "google/paligemma2-3b-pt-224"
torch_dtype: "float16"
max_new_tokens: 200
processor_path: "<required>"
model_cache_dir: "/home/ariel/.cache/sinapsis"
Inheritance
sinapsis_huggingface_transformers.templates.pali_gemma.pali_gemma_base.PaliGemmaBaseEnvironment variables
Environment definitions belong to their canonical Python modules.
No directly related environment variables listed.