Docs /Templates/ sinapsis-huggingface-transformers

ImageToTextTransformers

sinapsis-huggingface-transformersTransformersmodelsaudiospeechtext-to-speech

Python template · sinapsis_huggingface_transformers.templates.image_to_text_transformers

Overview

ImageToTextTransformers template to generate text from an image.

This template uses a Hugging Face Transformers pipeline to generate textual descriptions from input images.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-huggingface-transformers==1.0.0
uv pip install sinapsis-huggingface-transformers==1.0.0
uv add sinapsis-huggingface-transformers==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

seed
int | None
Optional
Default: null
metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
model_path
typing.Literal['microsoft/trocr-base-handwritten']
Optional
Default: "microsoft/trocr-base-handwritten"
torch_dtype
typing.Literal['float16', 'float32']
Optional
Default: "float16"
model_cache_dir
str
Optional
Default: "/home/ariel/.cache/sinapsis"
pipeline_kwargs
PipelineKwargs
Optional
Default: null
deviceRequired
typing.Literal['cuda', 'cpu']
Required
Default: null
torch_dtype
typing.Literal['float16', 'float32', 'auto']
Optional
Default: "float16"
inference_kwargs

Task-specific parameters for the image-to-text pipeline, such as `max_new_tokens`.

ImageToTextInferenceKwargs
Optional
Default: null
timeout

The maximum time in seconds to wait for fetching images from the web.

float | None
Optional
Default: null
max_new_tokens

The maximum number of tokens to generate in the description.

int | None
Optional
Default: null
generate_kwargs
dict[str, typing.Any] | None
Optional
Default: null

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: ImageToTextTransformers
  class_name: ImageToTextTransformers
  template_input: InputTemplate
  attributes:
    model_path: '/path/to/model'
    model_cache_dir: /path/to/cache/dir
    device: 'cuda'
    torch_dtype: float16
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "ImageToTextTransformers"
    class_name: "ImageToTextTransformers"
    attributes:
      seed: null
      model_path: "microsoft/trocr-base-handwritten"
      torch_dtype: "float16"
      model_cache_dir: "/home/ariel/.cache/sinapsis"
      pipeline_kwargs:
        device: "<required>"
        torch_dtype: "float16"
      inference_kwargs:
        timeout: null
        max_new_tokens: null
        generate_kwargs: null

Inheritance

Immediate parent: sinapsis_huggingface_transformers.templates.base_transformers.TransformersBase

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.