Docs /Templates/ sinapsis-vllm

vLLMMultiModal

sinapsis-vllmvLLMchatbotslarge_language_modelstext_completiontext

Python template · sinapsis_vllm.templates.vllm_multimodal

Overview

Template for multimodal (text + image) completion using vLLM.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-vllm==1.0.0
uv pip install sinapsis-vllm==1.0.0
uv add sinapsis-vllm==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
init_argsRequired

vLLM multimodal engine arguments.

vLLMMultimodalInitArgs
Required
Default: null
seed
int
Optional
Default: 0
dtype
typing.Literal['auto', 'half', 'float16', 'bfloat16', 'float', 'float32']
Optional
Default: "auto"
download_dir
str
Optional
Default: "/home/ariel/.cache/sinapsis"
max_num_seqs
int
Optional
Default: 256
quantization
typing.Optional[typing.Literal['awq', 'auto_awq', 'fp8', 'fbgemm_fp8', 'fp_quant', 'modelopt', 'modelopt_fp4', 'modelopt_mxfp8', 'modelopt_mixed', 'auto_gptq', 'gptq', 'gptq_marlin', 'awq_marlin', 'humming', 'compressed-tensors', 'experts_int8', 'quark', 'moe_wna16', 'torchao', 'inc', 'mxfp4', 'gpt_oss_mxfp4', 'deepseek_v4_fp8', 'online', 'fp8_per_tensor', 'fp8_per_block', 'fp8_per_channel', 'int8_per_channel_weight_only', 'nvfp4_per_token', 'mxfp8']]
Optional
Default: null
enforce_eager
bool
Optional
Default: false
max_model_len
int | None
Optional
Default: null
cpu_offload_gb
float
Optional
Default: 0
llm_model_nameRequired
str
Required
Default: null
tokenizer_mode
typing.Literal['auto', 'hf', 'alpamayo2_super', 'mistral', 'deepseek_v32', 'deepseek_v4', 'inkling', 'kimi_k3', 'cohere']
Optional
Default: "auto"
disable_log_stats
bool
Optional
Default: false
trust_remote_code

Whether to allow custom model code.

bool
Optional
Default: true
limit_mm_per_prompt

Per-prompt multimodal item limits.

dict[str, int]
Optional
Default: null
tensor_parallel_size
int
Optional
Default: 1
gpu_memory_utilization
float
Optional
Default: 0.9
completion_args
vLLMCompletionArgs
Optional
Default: null
seed

Optional per-request seed.

int | None
Optional
Default: null
stop

Stop sequences.

str | list[str] | None
Optional
Default: null
min_p

Minimum relative token probability.

float
Optional
Default: 0
top_k
int | None
Optional
Default: null
top_p
float | None
Optional
Default: null
bad_words

Words disallowed during generation.

list[str] | None
Optional
Default: null
ignore_eos

Whether EOS should be ignored.

bool
Optional
Default: false
max_tokens

Maximum number of generated tokens.

int
Optional
Default: 16
min_tokens

Minimum number of generated tokens.

int
Optional
Default: 0
temperature
float | None
Optional
Default: null
response_format

Structured output configuration.

vLLMResponseFormat
Optional
Default: null
type_
typing.Literal['text', 'json_object']
Optional
Default: "text"
schema_
SchemaDefinition
Optional
Default: null
required

Required root property names.

list[str]
Optional
Default: null
properties

Root schema properties.

dict[str, str | sinapsis_chatbots_base.helpers.schemas.PropertyDefinition]
Optional
Default: null
additional_properties

Whether extra root-level fields are allowed.

bool | None
Optional
Default: null
presence_penalty

Penalty for introducing new tokens.

float
Optional
Default: 0
frequency_penalty

Penalty based on token frequency.

float
Optional
Default: 0
repetition_penalty

Penalty based on token repetition.

float
Optional
Default: 1
cleanup_response
bool
Optional
Default: true
reasoning_end_tag
str | None
Optional
Default: "</think>"
reasoning_start_tag
str | None
Optional
Default: "<think>"

Configuration example

Example from the Python docstring:

yaml
agent:
      name: my_test_agent
    templates:
    - template_name: InputTemplate
      class_name: InputTemplate
      attributes: {}
    - template_name: vLLMMultiModal
      class_name: vLLMMultiModal
      template_input: InputTemplate
      attributes:
        init_args:
          llm_model_name: Qwen/Qwen2.5-VL-3B-Instruct
          max_model_len: 4096
          trust_remote_code: true
        completion_args:
          temperature: 0.2
          max_tokens: 512
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "vLLMMultiModal"
    class_name: "vLLMMultiModal"
    attributes:
      init_args:
        seed: 0
        dtype: "auto"
        download_dir: "/home/ariel/.cache/sinapsis"
        max_num_seqs: 256
        quantization: null
        enforce_eager: false
        max_model_len: null
        cpu_offload_gb: 0
        llm_model_name: "<required>"
        tokenizer_mode: "auto"
        disable_log_stats: false
        trust_remote_code: true
        limit_mm_per_prompt: null
        tensor_parallel_size: 1
        gpu_memory_utilization: 0.9
      completion_args:
        seed: null
        stop: null
        min_p: 0
        top_k: null
        top_p: null
        bad_words: null
        ignore_eos: false
        max_tokens: 16
        min_tokens: 0
        temperature: null
        response_format:
          type_: "text"
          schema_:
            required: null
            properties: null
            additional_properties: null
        presence_penalty: 0
        frequency_penalty: 0
        repetition_penalty: 1
      cleanup_response: true
      reasoning_end_tag: "</think>"
      reasoning_start_tag: "<think>"

Inheritance

Immediate parent: sinapsis_chatbots_base.templates.llm_multimodal_base.LLMMultimodalTemplateBase

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.