Docs /Templates/ sinapsis_langchain_readers

HuggingFaceDatasetLoaderWrapper

sinapsis_langchain_readersLangChaindocumentsdocument_loadingdynamicfiles

Python template · sinapsis_langchain_readers.templates.langchain_data_readers

Overview

Dynamic Template to load documents from Langchain document_loaders module The template loads the document either as a Document object in the generic_data field of DataContainer of each string as a TextPacket if add_document_as_text_packet is set as True

HuggingFaceDatasetLoader — inherited documentation

Load from `Hugging Face Hub` datasets.Initialize the HuggingFaceDatasetLoader.

Args: path: Path or name of the dataset. page_content_column: Page content column name. Default is "text". name: Name of the dataset configuration. data_dir: Data directory of the dataset configuration. data_files: Path(s) to source data file(s). cache_dir: Directory to read/write data. keep_in_memory: Whether to copy the dataset in-memory. save_infos: Save the dataset information (checksums/size/splits/...). Default is False. use_auth_token: Bearer token for remote files on the Dataset Hub. num_proc: Number of processes.

Capabilities

Capabilities: not set

Installation

Install the published package using pip or uv.

Terminalbash
pip install sinapsis_langchain_readers==1.0.0
uv pip install sinapsis_langchain_readers==1.0.0
uv add sinapsis_langchain_readers==1.0.0

Configuration

Effective fields from the inspector schema, including nested Pydantic model fields.

metadata
sinapsis_core.template_base.base_models.TemplateMetadata | None
Optional
Default: null
add_document_as_text_packet
Whether to add document as text packet or not. huggingfacedatasetloader_init path <class 'str'>: default: None page_content_column <class 'str'>: default: text name typing.Optional[str]: default: None data_dir… Read full field description

Whether to add document as text packet or not. huggingfacedatasetloader_init path <class 'str'>: default: None page_content_column <class 'str'>: default: text name typing.Optional[str]: default: None data_dir typing.Optional[str]: default: None data_files typing.Union[str, typing.Sequence[str], typing.Mapping[str, typing.Union[str, typing.Sequence[str]]], NoneType]: default: None cache_dir typing.Optional[str]: default: None keep_in_memory typing.Optional[bool]: default: None save_infos <class 'bool'>: default: False use_auth_token typing.Union[bool, str, NoneType]: default: None num_proc typing.Optional[int]: default: None

bool
Optional
Default: false
huggingfacedatasetloader_initRequired
__init__Attributes
Required
Default: null
name
typing.Optional[str]
Optional
Default: null
path
str
Optional
Default: null
data_dir
typing.Optional[str]
Optional
Default: null
num_proc
typing.Optional[int]
Optional
Default: null
cache_dir
typing.Optional[str]
Optional
Default: null
data_files
typing.Union[str, typing.Sequence[str], typing.Mapping[str, typing.Union[str, typing.Sequence[str]]], NoneType]
Optional
Default: null
save_infos
bool
Optional
Default: false
keep_in_memory
typing.Optional[bool]
Optional
Default: null
use_auth_token
typing.Union[bool, str, NoneType]
Optional
Default: null
page_content_column
str
Optional
Default: "text"

Configuration example

Generated from documented defaults. Replace any placeholders and complete required nested settings before running.

yaml
templates:
  - template_name: "HuggingFaceDatasetLoaderWrapper"
    class_name: "HuggingFaceDatasetLoaderWrapper"
    attributes:
      add_document_as_text_packet: false
      huggingfacedatasetloader_init:
        name: null
        path: null
        data_dir: null
        num_proc: null
        cache_dir: null
        data_files: null
        save_infos: false
        keep_in_memory: null
        use_auth_token: null
        page_content_column: "text"
View original docstring example

The source example references WikipediaLoaderWrapper, not HuggingFaceDatasetLoaderWrapper.

yaml
agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: WikipediaLoaderWrapper
  class_name: WikipediaLoaderWrapper    ## note that because it is a dynamic template,
  template_input: InputTemplate         ##  the class name depends on the actual class that is used.
  attributes:
    add_document_as_text_packet: false
    wikipedialoader_init:
        query: the query for wikipedia
        lang: en
        load_max_docs: 5000
        load_all_available_meta: False
        doc_content_chars_max: 4000

Inheritance

Immediate parent: sinapsis_langchain_readers.templates.langchain_data_readers.LangChainDataReaderBase

Environment variables

Environment definitions belong to their canonical Python modules.

SINAPSIS_BUILD_DOCS · defined elsewhere