Docs /Templates/ sinapsis-unsloth

UnslothInferenceReasoning

sinapsis-unslothUnsloth

Python template · sinapsis_unsloth.templates.inference.unsloth_inference_reasoning

Overview

Template for reasoning-based models (e.g., DeepSeek R1).

Extends conversational inference to support Chain-of-Thought (CoT) extraction.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-unsloth==1.0.0
uv pip install sinapsis-unsloth==1.0.0
uv add sinapsis-unsloth==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

stream
bool
Optional
Default: false
metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
model_argsRequired
UnslothModelArgs
Required
Default: null
dtype

Data type for model weights. Defaults to "auto".

typing.Literal['auto', 'bfloat16', 'float16']
Optional
Default: "auto"
cache_dir

Directory where models are cached. Defaults to SINAPSIS_CACHE_DIR.

str
Optional
Default: "/home/ariel/.cache/sinapsis"
device_map

Device mapping strategy for model parallelism. Defaults to "sequential".

str
Optional
Default: "sequential"
model_nameRequired

The name of the model to be used.

str
Required
Default: null
load_in_4bit

Whether to use 4-bit quantization. Defaults to True.

bool
Optional
Default: true
load_in_8bit

Whether to use 8-bit quantization. Defaults to False.

bool
Optional
Default: false
random_state

Random state for reproducibility. Defaults to 3407.

int
Optional
Default: 3407
load_in_16bit

Whether to use 16-bit quantization. Defaults to False.

bool
Optional
Default: false
max_lora_rank

Maximum rank for LoRA layers. Defaults to 64.

int
Optional
Default: 64
fast_inference

Whether to enable fast inference mode. Defaults to False.

bool
Optional
Default: false
max_seq_length

Maximum sequence length for input data. Defaults to 2048.

int
Optional
Default: 2048
full_finetuning

Whether to perform full fine-tuning. Defaults to False.

bool
Optional
Default: false
gpu_memory_utilization

Maximum GPU memory utilization as a fraction of total available memory. Defaults to 0.5.

float
Optional
Default: 0.5
use_gradient_checkpointing

Gradient checkpointing strategy. Defaults to "unsloth".

str
Optional
Default: "unsloth"
chat_configRequired

Configuration object containing conversational settings plus specific tags for reasoning blocks (e.g., <think>, </think>).

ReasoningChatConfig
Required
Default: null
mapping
ChatMapping
Optional
Default: null
role

Key name for the role field.

str
Optional
Default: "role"
user

Value identifying the user role.

str
Optional
Default: "user"
content

Key name for the content field.

str
Optional
Default: "content"
assistant

Value identifying the assistant role.

str
Optional
Default: "assistant"
chat_templateRequired
str
Required
Default: null
map_eos_token
bool
Optional
Default: true
system_message
str | None
Optional
Default: null
reasoning_open_tagRequired

The tag marking the start of the reasoning/thought block (e.g., '<think>').

str
Required
Default: null
reasoning_close_tagRequired

The tag marking the end of the reasoning/thought block (e.g., '</think>').

str
Required
Default: null
generate_argsRequired
UnslothGenerateArgs
Required
Default: null
min_p

Minimum probability for a token to be considered. Defaults to None.

float | None
Optional
Default: null
top_k

The number of highest probability vocabulary tokens to keep for top-k-filtering. Defaults to 50.

int
Optional
Default: 50
top_p

If set to float < 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation. Defaults to 1.0.

float
Optional
Default: 1
do_sample

Whether to use sampling; use greedy decoding otherwise. Defaults to True.

bool
Optional
Default: true
num_beams

Number of beams for beam search. 1 means no beam search. Defaults to 1.

int
Optional
Default: 1
use_cache

Whether to use key/value cache for faster generation. Defaults to True.

bool
Optional
Default: true
temperature

The value used to modulate the next token probabilities. 1.0 is neutral. Lower is more deterministic. Defaults to 1.0.

float
Optional
Default: 1
stop_strings

A string or list of strings that should stop generation when encountered. Defaults to None.

str | list[str] | None
Optional
Default: null
max_new_tokensRequired

The maximum number of tokens to generate.

int
Required
Default: null
min_new_tokens

The minimum number of tokens to generate. Defaults to None.

int | None
Optional
Default: null
repetition_penalty

The parameter for repetition penalty. 1.0 means no penalty. Defaults to 1.0.

float
Optional
Default: 1
cache_implementation

The cache implementation to use. "dynamic" is standard, "static" pre-allocates memory (faster but higher VRAM), "offloaded" moves cache to CPU. Defaults to "dynamic".

typing.Literal['dynamic', 'static', 'offloaded', 'offloaded_static', 'quantized']
Optional
Default: "dynamic"
rag_context_key
str | None
Optional
Default: null
chat_history_key
str | None
Optional
Default: null

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: UnslothInferenceReasoning
  class_name: UnslothInferenceReasoning
  template_input: InputTemplate
  attributes:
    model_args:
      model_name: unsloth/DeepSeek-R1-0528-Qwen3-8B
      max_seq_length: 2048
      dtype: auto
      load_in_4bit: true
      load_in_8bit: false
      load_in_16bit: false
      full_finetuning: false
      device_map: sequential
      use_gradient_checkpointing: unsloth
      fast_inference: false
      gpu_memory_utilization: 0.5
      random_state: 3407
      max_lora_rank: 64
    rag_context_key: null
    generate_args:
      max_new_tokens: 256
      min_new_tokens: null
      stop_strings: null
      do_sample: true
      num_beams: 1
      use_cache: true
      cache_implementation: dynamic
      temperature: 1.0
      top_k: 50
      top_p: 1.0
      min_p: null
      repetition_penalty: 1.0
    stream: false
    chat_history_key: null
    chat_config:
      mapping:
        role: role
        content: content
        user: user
        assistant: assistant
      map_eos_token: true
      system_message: null
      chat_template: deepseek-r1
      reasoning_open_tag: '<think>'
      reasoning_close_tag: '</think>'
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "UnslothInferenceReasoning"
    class_name: "UnslothInferenceReasoning"
    attributes:
      stream: false
      model_args:
        dtype: "auto"
        cache_dir: "/home/ariel/.cache/sinapsis"
        device_map: "sequential"
        model_name: "<required>"
        load_in_4bit: true
        load_in_8bit: false
        random_state: 3407
        load_in_16bit: false
        max_lora_rank: 64
        fast_inference: false
        max_seq_length: 2048
        full_finetuning: false
        gpu_memory_utilization: 0.5
        use_gradient_checkpointing: "unsloth"
      chat_config:
        mapping:
          role: "role"
          user: "user"
          content: "content"
          assistant: "assistant"
        chat_template: "<required>"
        map_eos_token: true
        system_message: null
        reasoning_open_tag: "<required>"
        reasoning_close_tag: "<required>"
      generate_args:
        min_p: null
        top_k: 50
        top_p: 1
        do_sample: true
        num_beams: 1
        use_cache: true
        temperature: 1
        stop_strings: null
        max_new_tokens: "<required>"
        min_new_tokens: null
        repetition_penalty: 1
        cache_implementation: "dynamic"
      rag_context_key: null
      chat_history_key: null

Inheritance

Immediate parent: sinapsis_unsloth.templates.inference.unsloth_inference_conversational.UnslothInferenceConversational

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.