vLLMStreamingTextCompletion
Python template · sinapsis_vllm.templates.vllm_streaming_text_completion
Overview
Streaming version of the vLLM text completion template.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis-vllm==1.0.0uv pip install sinapsis-vllm==1.0.0uv add sinapsis-vllm==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: null
vLLM engine configuration arguments.
Required
Default: null
Random seed used by the engine.
Optional
Default: 0
Weight and activation dtype.
Optional
Default: "auto"
Directory where weights are downloaded and loaded from.
Optional
Default: "/home/ariel/.cache/sinapsis"
Maximum concurrent sequences.
Optional
Default: 256
Quantization method, when used.
Optional
Default: null
Whether to disable CUDA graphs and force eager execution.
Optional
Default: false
Maximum model context length.
Optional
Default: null
CPU offload size in GB.
Optional
Default: 0
Required
Default: null
Tokenizer mode.
Optional
Default: "auto"
Whether to disable periodic runtime stats logging.
Optional
Default: false
Whether to allow custom model code.
Optional
Default: false
Number of GPUs used for tensor parallelism.
Optional
Default: 1
Fraction of GPU memory reserved for the model.
Optional
Default: 0.9
vLLM sampling and generation arguments.
Optional
Default: null
Optional per-request seed.
Optional
Default: null
Stop sequences.
Optional
Default: null
Minimum relative token probability.
Optional
Default: 0
Optional
Default: null
Optional
Default: null
Words disallowed during generation.
Optional
Default: null
Whether EOS should be ignored.
Optional
Default: false
Maximum number of generated tokens.
Optional
Default: 16
Minimum number of generated tokens.
Optional
Default: 0
Optional
Default: null
Structured output configuration.
Optional
Default: null
Optional
Default: "text"
Optional
Default: null
Required root property names.
Optional
Default: null
Root schema properties.
Optional
Default: null
Whether extra root-level fields are allowed.
Optional
Default: null
Penalty for introducing new tokens.
Optional
Default: 0
Penalty based on token frequency.
Optional
Default: 0
Penalty based on token repetition.
Optional
Default: 1
Optional
Default: true
Optional
Default: "</think>"
Optional
Default: "<think>"
Configuration example
Example from the Python docstring:
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: vLLMStreamingTextCompletion
class_name: vLLMStreamingTextCompletion
template_input: InputTemplate
attributes:
init_args:
llm_model_name: Qwen/Qwen3-1.7B
max_model_len: 4096
completion_args:
temperature: 0.7
max_tokens: 512Show schema-based configuration skeleton
templates:
- template_name: "vLLMStreamingTextCompletion"
class_name: "vLLMStreamingTextCompletion"
attributes:
init_args:
seed: 0
dtype: "auto"
download_dir: "/home/ariel/.cache/sinapsis"
max_num_seqs: 256
quantization: null
enforce_eager: false
max_model_len: null
cpu_offload_gb: 0
llm_model_name: "<required>"
tokenizer_mode: "auto"
disable_log_stats: false
trust_remote_code: false
tensor_parallel_size: 1
gpu_memory_utilization: 0.9
completion_args:
seed: null
stop: null
min_p: 0
top_k: null
top_p: null
bad_words: null
ignore_eos: false
max_tokens: 16
min_tokens: 0
temperature: null
response_format:
type_: "text"
schema_:
required: null
properties: null
additional_properties: null
presence_penalty: 0
frequency_penalty: 0
repetition_penalty: 1
cleanup_response: true
reasoning_end_tag: "</think>"
reasoning_start_tag: "<think>"
Inheritance
sinapsis_chatbots_base.templates.llm_streaming_base.LLMStreamingTemplateBaseEnvironment variables
Environment definitions belong to their canonical Python modules.