haive.core.models.llm.providers.replicate

Replicate Provider Module.

This module implements the Replicate language model provider for the Haive framework, supporting a wide variety of open-source models hosted on Replicate’s platform.

The provider handles API key management, model configuration, and safe imports of the langchain-community package dependencies for Replicate integration.

Examples

Basic usage:

from haive.core.models.llm.providers.replicate import ReplicateProvider

provider = ReplicateProvider(
    model="meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3",
    temperature=0.7,
    max_tokens=1000
)
llm = provider.instantiate()

With custom parameters:

provider = ReplicateProvider(
    model="mistralai/mixtral-8x7b-instruct-v0.1",
    temperature=0.1,
    top_p=0.9,
    top_k=50
)

Classes

ReplicateProvider

Replicate language model provider configuration.

Module Contents

class haive.core.models.llm.providers.replicate.ReplicateProvider(/, **data)

Bases: haive.core.models.llm.providers.base.BaseLLMProvider

Replicate language model provider configuration.

This provider supports a wide variety of open-source models hosted on Replicate, including Llama, Mixtral, CodeLlama, and many others with flexible versioning.

Parameters:

data (Any)

provider

Always LLMProvider.REPLICATE

Type:

LLMProvider

model

The Replicate model to use (owner/name:version format)

Type:

str

temperature

Sampling temperature (0.0-5.0)

Type:

float

max_tokens

Maximum tokens in response

Type:

int

top_p

Nucleus sampling parameter

Type:

float

top_k

Top-k sampling parameter

Type:

int

repetition_penalty

Repetition penalty parameter

Type:

float

stop_sequences

Stop sequences for generation

Type:

list

Examples

Llama 2 70B model:

provider = ReplicateProvider(
    model="meta/llama-2-70b-chat",
    temperature=0.7,
    max_tokens=2000
)

Mixtral with specific version:

provider = ReplicateProvider(
    model="mistralai/mixtral-8x7b-instruct-v0.1:7b3212fbaf88310047672c7764d9f2cce7493d0d80666d899b72af8c0662df7a",
    temperature=0.1,
    top_p=0.9
)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

classmethod get_models()

Get popular Replicate models.

Return type:

list[str]

classmethod validate_model_format(v)

Validate Replicate model format.

Parameters:

v (str)

Return type:

str

max_tokens: int | None = None

Get maximum total tokens for this model.