LLaMACPPTextCompletionWithMCP
Python template · sinapsis_llama_cpp.templates.llama_cpp_text_completion_mcp
Overview
Run llama-cpp chat completion with packet-native MCP tool support.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis-llama-cpp==1.0.0uv pip install sinapsis-llama-cpp==1.0.0uv add sinapsis-llama-cpp==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: null
Llama runtime initialization arguments.
Required
Default: null
Base RNG seed. Defaults to `LLAMA_DEFAULT_SEED`.
Optional
Default: 4294967295
Context window size. Defaults to `512`.
Optional
Default: 512
Prompt/prefill batch size. Defaults to `2048`.
Optional
Default: 2048
Verbose llama.cpp logging. Defaults to `True`.
Optional
Default: true
Primary GPU index. Defaults to `0`.
Optional
Default: 0
Micro-batch size. Defaults to `512`.
Optional
Default: 512
Whether to memory-map the model file. Defaults to `True`.
Optional
Default: true
Optional base model path for LoRA setups. Defaults to `None`.
Optional
Default: null
LoRA adapter path. Defaults to `None`.
Optional
Default: null
CPU threads for generation. Defaults to `None`.
Optional
Default: null
Whether to lock the model in RAM. Defaults to `False`.
Optional
Default: false
LoRA strength. Defaults to `1.0`.
Optional
Default: 1
Multi-GPU split mode (0=none, 1=row, 2=layer). Defaults to `0`.
Optional
Default: 0
Chat template id used by `create_chat_completion`. Defaults to `None`.
Optional
Default: null
Whether to offload KV/attention tensors to GPU. Defaults to `False`.
Optional
Default: false
Layers to offload to GPU (0=CPU, -1=all). Defaults to `0`.
Optional
Default: 0
Per-GPU split fractions. Defaults to `None`.
Optional
Default: null
GGUF filename to load.
Required
Default: null
Model repo/name or local identifier to load from.
Required
Default: null
Flash-attention policy. Defaults to `-1`.
Optional
Default: -1
CPU threads for prompt eval. Defaults to `None`.
Optional
Default: null
Llama generation arguments.
Optional
Default: null
Per-request seed override. Defaults to `None`.
Optional
Default: null
Stop sequence(s). Defaults to `None`.
Optional
Default: null
Minimum probability threshold. Defaults to `0.05`.
Optional
Default: 0.05
Optional
Default: null
Optional
Default: null
Typical sampling value. Defaults to `1.0`.
Optional
Default: 1
Token logit bias overrides. Defaults to `None`.
Optional
Default: null
Maximum number of output tokens.
Required
Default: null
Optional
Default: null
Repetition window. Defaults to `64`.
Optional
Default: 64
Repetition penalty. Defaults to `1.0`.
Optional
Default: 1
Presence penalty. Defaults to `0.0`.
Optional
Default: 0
LLaMA structured-output configuration.
Optional
Default: null
Optional
Default: "text"
Optional
Default: null
Required root property names.
Optional
Default: null
Root schema properties.
Optional
Default: null
Whether extra root-level fields are allowed.
Optional
Default: null
Frequency penalty. Defaults to `0.0`.
Optional
Default: 0
Optional
Default: true
Maximum consecutive tool failures before stopping the loop.
Optional
Default: 3
Optional
Default: "</think>"
Whether to append tool descriptions to the system prompt.
Optional
Default: true
Optional
Default: "<think>"
Configuration example
Example from the Python docstring:
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: LLaMACPPTextCompletionWithMCP
class_name: LLaMACPPTextCompletionWithMCP
template_input: InputTemplate
attributes:
init_args:
llm_model_name: bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF
llm_model_file: DeepSeek-R1-Distill-Qwen-7B-Q4_K_S.gguf
chat_format: chatml-function-calling
n_ctx: 8192
n_threads: 8
n_gpu_layers: 15
completion_args:
max_tokens: 2048
temperature: 0.2
max_tool_retries: 3
add_tool_to_prompt: trueShow schema-based configuration skeleton
templates:
- template_name: "LLaMACPPTextCompletionWithMCP"
class_name: "LLaMACPPTextCompletionWithMCP"
attributes:
init_args:
seed: 4294967295
n_ctx: 512
n_batch: 2048
verbose: true
main_gpu: 0
n_ubatch: 512
use_mmap: true
lora_base: null
lora_path: null
n_threads: null
use_mlock: false
lora_scale: 1
split_mode: 0
chat_format: null
offload_kqv: false
n_gpu_layers: 0
tensor_split: null
llm_model_file: "<required>"
llm_model_name: "<required>"
flash_attn_type: -1
n_threads_batch: null
completion_args:
seed: null
stop: null
min_p: 0.05
top_k: null
top_p: null
typical_p: 1
logit_bias: null
max_tokens: "<required>"
temperature: null
penalty_last_n: 64
repeat_penalty: 1
present_penalty: 0
response_format:
type_: "text"
schema_:
required: null
properties: null
additional_properties: null
frequency_penalty: 0
cleanup_response: true
max_tool_retries: 3
reasoning_end_tag: "</think>"
add_tool_to_prompt: true
reasoning_start_tag: "<think>"
Inheritance
sinapsis_llama_cpp.templates.llama_cpp_text_completion.LLaMACPPTextCompletionEnvironment variables
Environment definitions belong to their canonical Python modules.
No directly related environment variables listed.