haive.core.engine.document.loaders.adapters.base

Base loader adapter for document loaders.

This module provides the base LoaderAdapter class that all specific loader adapters inherit from, establishing a consistent interface.

Classes

LoaderAdapter

Base adapter class for document loaders.

Module Contents

class haive.core.engine.document.loaders.adapters.base.LoaderAdapter(source, **params)

Bases: abc.ABC

Base adapter class for document loaders.

LoaderAdapter provides a unified interface for loading documents from different source types using various langchain document loaders.

Each adapter is responsible for: 1. Creating the appropriate loader for a specific source type 2. Configuring the loader with the correct parameters 3. Loading documents from the source 4. Optionally, implementing fetch_all functionality when supported

Initialize the adapter.

Parameters:

• source (haive.core.engine.document.loaders.sources.base.base.BaseSource) – The source to load documents from

• **params – Additional parameters for the loader

abstractmethod fetch_all()

Fetch all available sources (if supported).

This method is used for sources that contain multiple sub-sources, such as directories or sitemaps.

Returns:

List of individual sources

Raises:

NotImplementedError – If the adapter doesn’t support fetch_all

Return type:

list[haive.core.engine.document.loaders.sources.base.base.BaseSource]

abstractmethod load()

Load documents from the source.

This method must be implemented by all subclasses.

Returns:

List of loaded documents

Return type:

list[langchain_core.documents.Document]

load_and_split(**split_params)

Load and split documents.

This method loads documents and then splits them into chunks using a text splitter.

Parameters:

**split_params – Parameters for the text splitter

Returns:

List of document chunks

Return type:

list[langchain_core.documents.Document]