Docs /Templates/ sinapsis-unsloth

UnslothExportMerged

sinapsis-unslothUnsloth

Python template · sinapsis_unsloth.templates.export.unsloth_export_merged

Overview

Template for exporting merged models with Unsloth.

Provides functionality to export Unsloth-optimized models either locally or to the Hugging Face Hub.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis-unsloth==1.0.0
uv pip install sinapsis-unsloth==1.0.0
uv add sinapsis-unsloth==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
model_argsRequired
UnslothModelArgs
Required
Default: null
dtype

Data type for model weights. Defaults to "auto".

typing.Literal['auto', 'bfloat16', 'float16']
Optional
Default: "auto"
cache_dir

Directory where models are cached. Defaults to SINAPSIS_CACHE_DIR.

str
Optional
Default: "/home/ariel/.cache/sinapsis"
device_map

Device mapping strategy for model parallelism. Defaults to "sequential".

str
Optional
Default: "sequential"
model_nameRequired

The name of the model to be used.

str
Required
Default: null
load_in_4bit

Whether to use 4-bit quantization. Defaults to True.

bool
Optional
Default: true
load_in_8bit

Whether to use 8-bit quantization. Defaults to False.

bool
Optional
Default: false
random_state

Random state for reproducibility. Defaults to 3407.

int
Optional
Default: 3407
load_in_16bit

Whether to use 16-bit quantization. Defaults to False.

bool
Optional
Default: false
max_lora_rank

Maximum rank for LoRA layers. Defaults to 64.

int
Optional
Default: 64
fast_inference

Whether to enable fast inference mode. Defaults to False.

bool
Optional
Default: false
max_seq_length

Maximum sequence length for input data. Defaults to 2048.

int
Optional
Default: 2048
full_finetuning

Whether to perform full fine-tuning. Defaults to False.

bool
Optional
Default: false
gpu_memory_utilization

Maximum GPU memory utilization as a fraction of total available memory. Defaults to 0.5.

float
Optional
Default: 0.5
use_gradient_checkpointing

Gradient checkpointing strategy. Defaults to "unsloth".

str
Optional
Default: "unsloth"
export_argsRequired

Configuration arguments for exporting the merged model.

UnslothExportMergedArgs
Required
Default: null
save_pathRequired
str
Required
Default: null
save_method

The method to use for saving the export. Defaults to 'merged_16bit'.

str
Optional
Default: "merged_16bit"
max_shard_size
str | int
Optional
Default: "5GB"
maximum_memory_usage
float
Optional
Default: 0.75
push_to_hub
bool
Optional
Default: false

Configuration example

Example from the Python docstring:

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: UnslothExportMerged
  class_name: UnslothExportMerged
  template_input: InputTemplate
  attributes:
    model_args:
      model_name: unsloth/DeepSeek-R1-Distill-Qwen-1.5B
      max_seq_length: 2048
      dtype: auto
      load_in_4bit: true
      load_in_8bit: false
      load_in_16bit: false
      full_finetuning: false
      device_map: sequential
      use_gradient_checkpointing: unsloth
      fast_inference: false
      gpu_memory_utilization: 0.5
      random_state: 3407
      max_lora_rank: 64
    export_args:
      save_path: artifacts/DeepSeek-R1-Distill-Qwen-1.5B-16-bit
      max_shard_size: 5GB
      maximum_memory_usage: 0.75
      save_method: merged_16bit
    push_to_hub: false
Show schema-based configuration skeleton
yaml
templates:
  - template_name: "UnslothExportMerged"
    class_name: "UnslothExportMerged"
    attributes:
      model_args:
        dtype: "auto"
        cache_dir: "/home/ariel/.cache/sinapsis"
        device_map: "sequential"
        model_name: "<required>"
        load_in_4bit: true
        load_in_8bit: false
        random_state: 3407
        load_in_16bit: false
        max_lora_rank: 64
        fast_inference: false
        max_seq_length: 2048
        full_finetuning: false
        gpu_memory_utilization: 0.5
        use_gradient_checkpointing: "unsloth"
      export_args:
        save_path: "<required>"
        save_method: "merged_16bit"
        max_shard_size: "5GB"
        maximum_memory_usage: 0.75
      push_to_hub: false

Inheritance

Immediate parent: sinapsis_unsloth.templates.export.unsloth_export_base.UnslothExportBase

Environment variables

Environment definitions belong to their canonical Python modules.

No directly related environment variables listed.