HuggingFaceDatasetLoaderWrapper
Python template · sinapsis_langchain_readers.templates.langchain_data_readers
Overview
Dynamic Template to load documents from Langchain document_loaders module The template loads the document either as a Document object in the generic_data field of DataContainer of each string as a TextPacket if add_document_as_text_packet is set as True
HuggingFaceDatasetLoader — inherited documentation
Load from `Hugging Face Hub` datasets.Initialize the HuggingFaceDatasetLoader.
Args: path: Path or name of the dataset. page_content_column: Page content column name. Default is "text". name: Name of the dataset configuration. data_dir: Data directory of the dataset configuration. data_files: Path(s) to source data file(s). cache_dir: Directory to read/write data. keep_in_memory: Whether to copy the dataset in-memory. save_infos: Save the dataset information (checksums/size/splits/...). Default is False. use_auth_token: Bearer token for remote files on the Dataset Hub. num_proc: Number of processes.
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis_langchain_readers==1.0.0uv pip install sinapsis_langchain_readers==1.0.0uv add sinapsis_langchain_readers==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: null
Whether to add document as text packet or not. huggingfacedatasetloader_init path <class 'str'>: default: None page_content_column <class 'str'>: default: text name typing.Optional[str]: default: None data_dir… Read full field description
Whether to add document as text packet or not. huggingfacedatasetloader_init path <class 'str'>: default: None page_content_column <class 'str'>: default: text name typing.Optional[str]: default: None data_dir typing.Optional[str]: default: None data_files typing.Union[str, typing.Sequence[str], typing.Mapping[str, typing.Union[str, typing.Sequence[str]]], NoneType]: default: None cache_dir typing.Optional[str]: default: None keep_in_memory typing.Optional[bool]: default: None save_infos <class 'bool'>: default: False use_auth_token typing.Union[bool, str, NoneType]: default: None num_proc typing.Optional[int]: default: None
Optional
Default: false
Required
Default: null
Optional
Default: null
Optional
Default: null
Optional
Default: null
Optional
Default: null
Optional
Default: null
Optional
Default: null
Optional
Default: false
Optional
Default: null
Optional
Default: null
Optional
Default: "text"
Configuration example
Generated from documented defaults. Replace any placeholders and complete required nested settings before running.
templates:
- template_name: "HuggingFaceDatasetLoaderWrapper"
class_name: "HuggingFaceDatasetLoaderWrapper"
attributes:
add_document_as_text_packet: false
huggingfacedatasetloader_init:
name: null
path: null
data_dir: null
num_proc: null
cache_dir: null
data_files: null
save_infos: false
keep_in_memory: null
use_auth_token: null
page_content_column: "text"
View original docstring example
The source example references WikipediaLoaderWrapper, not HuggingFaceDatasetLoaderWrapper.
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: WikipediaLoaderWrapper
class_name: WikipediaLoaderWrapper ## note that because it is a dynamic template,
template_input: InputTemplate ## the class name depends on the actual class that is used.
attributes:
add_document_as_text_packet: false
wikipedialoader_init:
query: the query for wikipedia
lang: en
load_max_docs: 5000
load_all_available_meta: False
doc_content_chars_max: 4000Inheritance
sinapsis_langchain_readers.templates.langchain_data_readers.LangChainDataReaderBaseEnvironment variables
Environment definitions belong to their canonical Python modules.