haive.core.models.llm.providers.nvidia¶
NVIDIA AI Endpoints Provider Module.
This module implements the NVIDIA AI Endpoints language model provider for the Haive framework, supporting NVIDIA’s optimized models through their AI Foundation API.
The provider handles API key management, model configuration, and safe imports of the langchain-nvidia-ai-endpoints package dependencies.
Examples
Basic usage:
from haive.core.models.llm.providers.nvidia import NVIDIAProvider
provider = NVIDIAProvider(
model="meta/llama3-70b-instruct",
temperature=0.7,
max_tokens=1000
)
llm = provider.instantiate()
With streaming:
provider = NVIDIAProvider(
model="microsoft/phi-3-medium-4k-instruct",
temperature=0.1,
stream=True
)
Classes¶
NVIDIA AI Endpoints language model provider configuration. |
Module Contents¶
- class haive.core.models.llm.providers.nvidia.NVIDIAProvider(/, **data)¶
Bases:
haive.core.models.llm.providers.base.BaseLLMProviderNVIDIA AI Endpoints language model provider configuration.
This provider supports NVIDIA’s optimized models including Llama, Mixtral, and other high-performance models through NVIDIA’s AI Foundation API.
- Parameters:
data (Any)
- provider¶
Always LLMProvider.NVIDIA
- Type:
Examples
Llama 3 for reasoning:
provider = NVIDIAProvider( model="meta/llama3-70b-instruct", temperature=0.3, max_tokens=2000 )
Mixtral for fast inference:
provider = NVIDIAProvider( model="mistralai/mixtral-8x22b-instruct-v0.1", temperature=0.7, stream=True )
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
self is explicitly positional-only to allow self as a field name.