ArxivLoaderExecuteNTimes
Python template · sinapsis_langchain_readers.templates.langchain_data_readers
Overview
Dynamic Template to load documents from Langchain document_loaders module The template loads the document either as a Document object in the generic_data field of DataContainer of each string as a TextPacket if add_document_as_text_packet is set as True
ArxivLoader — inherited documentation
Load a query result from `Arxiv`. The loader converts the original PDF format into the text.
Setup: Install ``arxiv`` and ``PyMuPDF`` packages. ``PyMuPDF`` transforms PDF files downloaded from the arxiv.org site into the text format.
.. code-block:: bash
pip install -U arxiv pymupdf
Instantiate: .. code-block:: python
from langchain_community.document_loaders import ArxivLoader
loader = ArxivLoader( query="reasoning", # load_max_docs=2, # load_all_available_meta=False )
Load: .. code-block:: python
docs = loader.load() print(docs[0].page_content[:100]) print(docs[0].metadata)
.. code-block:: python Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggre { 'Published': '2024-02-29', 'Title': 'Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation', 'Authors': 'Xinyi Wang, Alfonso Amayuelas, Kexun Zhang, Liangming Pan, Wenhu Chen, William Yang Wang', 'Summary': 'Pre-trained language models (LMs) are able to perform complex reasoning without explicit fine-tuning...' }
Lazy load: .. code-block:: python
docs = [] docs_lazy = loader.lazy_load()
# async variant: # docs_lazy = await loader.alazy_load()
for doc in docs_lazy: docs.append(doc) print(docs[0].page_content[:100]) print(docs[0].metadata)
.. code-block:: python
Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggre { 'Published': '2024-02-29', 'Title': 'Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation', 'Authors': 'Xinyi Wang, Alfonso Amayuelas, Kexun Zhang, Liangming Pan, Wenhu Chen, William Yang Wang', 'Summary': 'Pre-trained language models (LMs) are able to perform complex reasoning without explicit fine-tuning...' }
Async load: .. code-block:: python
docs = await loader.aload() print(docs[0].page_content[:100]) print(docs[0].metadata)
.. code-block:: python
Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggre { 'Published': '2024-02-29', 'Title': 'Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation', 'Authors': 'Xinyi Wang, Alfonso Amayuelas, Kexun Zhang, Liangming Pan, Wenhu Chen, William Yang Wang', 'Summary': 'Pre-trained language models (LMs) are able to perform complex reasoning without explicit fine-tuning...' }
Use summaries of articles as docs: .. code-block:: python
from langchain_community.document_loaders import ArxivLoader
loader = ArxivLoader( query="reasoning" )
docs = loader.get_summaries_as_docs() print(docs[0].page_content[:100]) print(docs[0].metadata)
.. code-block:: python
Pre-trained language models (LMs) are able to perform complex reasoning without explicit fine-tuning { 'Entry ID': 'http://arxiv.org/abs/2402.03268v2', 'Published': datetime.date(2024, 2, 29), 'Title': 'Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation', 'Authors': 'Xinyi Wang, Alfonso Amayuelas, Kexun Zhang, Liangming Pan, Wenhu Chen, William Yang Wang' }Initialize with search query to find documents in the Arxiv. Supports all arguments of `ArxivAPIWrapper`.
Args: query: free text which used to find documents in the Arxiv doc_content_chars_max: cut limit for the length of a document's content
Capabilities
Capabilities: not setInstallation
Install the published package using pip or uv.
pip install sinapsis_langchain_readers==1.0.0uv pip install sinapsis_langchain_readers==1.0.0uv add sinapsis_langchain_readers==1.0.0Configuration
Effective fields from the inspector schema, including nested Pydantic model fields.
Optional
Default: null
Optional
Default: 1
Required
Default: null
Optional
Default: false
Configuration example
Generated from documented defaults. Replace any placeholders and complete required nested settings before running.
templates:
- template_name: "ArxivLoaderExecuteNTimes"
class_name: "ArxivLoaderExecuteNTimes"
attributes:
n_times_execute: 1
arxivloader_init: {}
add_document_as_text_packet: false
View original docstring example
The source example references WikipediaLoaderWrapper, not ArxivLoaderExecuteNTimes.
agent:
name: my_test_agent
templates:
- template_name: InputTemplate
class_name: InputTemplate
attributes: {}
- template_name: WikipediaLoaderWrapper
class_name: WikipediaLoaderWrapper ## note that because it is a dynamic template,
template_input: InputTemplate ## the class name depends on the actual class that is used.
attributes:
add_document_as_text_packet: false
wikipedialoader_init:
query: the query for wikipedia
lang: en
load_max_docs: 5000
load_all_available_meta: False
doc_content_chars_max: 4000,
n_times_execute determines the number of times the template's execute
method is called. By default, it only gets called once. Further calls
to execute are bypassed.Inheritance
sinapsis_langchain_readers.templates.langchain_data_readers.ExecuteNTimesLangchainDataReadersEnvironment variables
Environment definitions belong to their canonical Python modules.