vLLMMultiModal
Python template · sinapsis_vllm.templates.vllm_multimodal
Overview
Template for multimodal (text + image) completion using vLLM.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis-vllm==1.0.0uv pip install sinapsis-vllm==1.0.0uv add sinapsis-vllm==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: null
vLLM multimodal engine arguments.
Required
Default: null
Optional
Default: 0
Optional
Default: "auto"
Optional
Default: "/home/ariel/.cache/sinapsis"
Optional
Default: 256
Optional
Default: null
Optional
Default: false
Optional
Default: null
Optional
Default: 0
Required
Default: null
Optional
Default: "auto"
Optional
Default: false
Whether to allow custom model code.
Optional
Default: true
Per-prompt multimodal item limits.
Optional
Default: null
Optional
Default: 1
Optional
Default: 0.9
Optional
Default: null
Optional per-request seed.
Optional
Default: null
Stop sequences.
Optional
Default: null
Minimum relative token probability.
Optional
Default: 0
Optional
Default: null
Optional
Default: null
Words disallowed during generation.
Optional
Default: null
Whether EOS should be ignored.
Optional
Default: false
Maximum number of generated tokens.
Optional
Default: 16
Minimum number of generated tokens.
Optional
Default: 0
Optional
Default: null
Structured output configuration.
Optional
Default: null
Optional
Default: "text"
Optional
Default: null
Required root property names.
Optional
Default: null
Root schema properties.
Optional
Default: null
Whether extra root-level fields are allowed.
Optional
Default: null
Penalty for introducing new tokens.
Optional
Default: 0
Penalty based on token frequency.
Optional
Default: 0
Penalty based on token repetition.
Optional
Default: 1
Optional
Default: true
Optional
Default: "</think>"
Optional
Default: "<think>"
Configuration example
Example from the Python docstring:
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: vLLMMultiModal
class_name: vLLMMultiModal
template_input: InputTemplate
attributes:
init_args:
llm_model_name: Qwen/Qwen2.5-VL-3B-Instruct
max_model_len: 4096
trust_remote_code: true
completion_args:
temperature: 0.2
max_tokens: 512Show schema-based configuration skeleton
templates:
- template_name: "vLLMMultiModal"
class_name: "vLLMMultiModal"
attributes:
init_args:
seed: 0
dtype: "auto"
download_dir: "/home/ariel/.cache/sinapsis"
max_num_seqs: 256
quantization: null
enforce_eager: false
max_model_len: null
cpu_offload_gb: 0
llm_model_name: "<required>"
tokenizer_mode: "auto"
disable_log_stats: false
trust_remote_code: true
limit_mm_per_prompt: null
tensor_parallel_size: 1
gpu_memory_utilization: 0.9
completion_args:
seed: null
stop: null
min_p: 0
top_k: null
top_p: null
bad_words: null
ignore_eos: false
max_tokens: 16
min_tokens: 0
temperature: null
response_format:
type_: "text"
schema_:
required: null
properties: null
additional_properties: null
presence_penalty: 0
frequency_penalty: 0
repetition_penalty: 1
cleanup_response: true
reasoning_end_tag: "</think>"
reasoning_start_tag: "<think>"
Inheritance
sinapsis_chatbots_base.templates.llm_multimodal_base.LLMMultimodalTemplateBaseEnvironment variables
Environment definitions belong to their canonical Python modules.
No directly related environment variables listed.