viruPVS/langchain-docs huggingface dataset:
langchain_community.llms.gpt4all.GPT4All¶
class langchain_community.llms.gpt4all.GPT4All[source]¶
Bases: LLM
GPT4All language models.
To use, you should have the gpt4all python package installed, the
pre-trained model file, and the model’s config information.
Example
from langchain_community.llms import GPT4All
model = GPT4All(model="./models/gpt4all-model.bin", n_threads=8)
# Simplest invocation
response = model("Once upon a time, ")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param allow_download: bool = False¶
If model does not exist in ~/.cache/gpt4all/, download it.
param backend: Optional[str] = None¶
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param device: Optional[str] = 'cpu'¶
Device name: cpu, gpu, nvidia, intel, amd or DeviceName.
param echo: Optional[bool] = False¶
Whether to echo the prompt.
param embedding: bool = False¶
Use embedding mode only.
param f16_kv: bool = False¶
Use half-precision for key/value cache.
param logits_all: bool = False¶
Return logits for all tokens, not just the last token.
param max_tokens: int = 200¶
Token context window.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str [Required]¶
Path to the pre-trained GPT4All model file.
Path to the pre-trained GPT4All model file.
param n_batch: int = 8¶
Batch size for prompt processing.
param n_parts: int = -1¶
Number of parts to split the model into.
If -1, the number of parts is automatically determined.
param n_predict: Optional[int] = 256¶
The maximum number of tokens to generate.
param n_threads: Optional[int] = 4¶
Number of threads to use.
param repeat_last_n: Optional[int] = 64¶
Last n tokens to penalize
param repeat_penalty: Optional[float] = 1.18¶
The penalty to apply to repeated tokens.
param seed: int = 0¶
Seed. If -1, a random seed is used.
param stop: Optional[List[str]] = []¶
A list of strings to stop generation when encountered.
param streaming: bool = False¶
Whether to stream the results or not.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temp: Optional[float] = 0.7¶
The temperature to use for sampling.
param top_k: Optional[int] = 40¶
The top-k value to use for sampling.
param top_p: Optional[float] = 0.1¶
The top-p value to use for sampling.
param use_mlock: bool = False¶
Force system to keep model in RAM.
param verbose: bool [Optional]¶
Whether to print out response text.
param vocab_only: bool = False¶
Only load the vocabulary, no weights.
param vocab_only: bool = False¶
Only load the vocabulary, no weights.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
async abatch(inputs: List[Union[PromptValue, str, List[BaseMessage]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
async agenerate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, **kwargs: Any) → LLMResult¶
Run the LLM on the given prompt and input.
between retries
stop_after_attempt – The maximum number of attempts to make before giving up
Returns
A new Runnable that retries the original runnable on exceptions.
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using GPT4All¶
PromptLayer
GPT4All
Run LLMs locally
Use local LLMs
langchain_community.llms.predictionguard.PredictionGuard¶
class langchain_community.llms.predictionguard.PredictionGuard[source]¶
Bases: LLM
Prediction Guard large language models.
To use, you should have the predictionguard python package installed, and the
environment variable PREDICTIONGUARD_TOKEN set with your access token, or pass
it as a named parameter to the constructor. To use Prediction Guard’s API along
with OpenAI models, set the environment variable OPENAI_API_KEY with your
OpenAI API key as well.
Example
pgllm = PredictionGuard(model="MPT-7B-Instruct",
                        token="my-access-token",
                        output={
                            "type": "boolean"
                        })
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param max_tokens: int = 256¶
Denotes the number of tokens to predict per generation.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: Optional[str] = 'MPT-7B-Instruct'¶
Model name to use.
param output: Optional[Dict[str, Any]] = None¶
The output type or structure for controlling the LLM output.
param stop: Optional[List[str]] = None¶
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.75¶
A non-negative float that tunes the degree of randomness in generation.
param token: Optional[str] = None¶
Your Prediction Guard access token.
param token: Optional[str] = None¶
Your Prediction Guard access token.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
async abatch(inputs: List[Union[PromptValue, str, List[BaseMessage]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
async agenerate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, **kwargs: Any) → LLMResult¶
Run the LLM on the given prompt and input.
between retries
stop_after_attempt – The maximum number of attempts to make before giving up
Returns
A new Runnable that retries the original runnable on exceptions.
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using PredictionGuard¶
Prediction Guard
langchain_community.llms.sagemaker_endpoint.LLMContentHandler¶
class langchain_community.llms.sagemaker_endpoint.LLMContentHandler[source]¶
Content handler for LLM class.
Attributes
accepts
The MIME type of the response data returned from endpoint
content_type
The MIME type of the input data passed to endpoint
Methods
__init__()
transform_input(prompt, model_kwargs)
Transforms the input to a format that model can accept as the request Body.
transform_output(output)
Transforms the output from the model to string that the LLM class expects.
__init__()¶
abstract transform_input(prompt: INPUT_TYPE, model_kwargs: Dict) → bytes¶
Transforms the input to a format that model can accept
as the request Body. Should return bytes or seekable file
like object in the format specified in the content_type
request header.
abstract transform_output(output: bytes) → OUTPUT_TYPE¶
Transforms the output from the model to string that
the LLM class expects.
Examples using LLMContentHandler¶
SageMaker Endpoint
SageMakerEndpoint
Amazon Comprehend Moderation Chain
langchain_community.llms.koboldai.KoboldApiLLM¶
class langchain_community.llms.koboldai.KoboldApiLLM[source]¶
Bases: LLM
Kobold API language model.
It includes several fields that can be used to control the text generation process.
To use this class, instantiate it with the required parameters and call it with a
prompt to generate text. For example:
kobold = KoboldApiLLM(endpoint=”http://localhost:5000”)
result = kobold(“Write a story about a dragon.”)
This will send a POST request to the Kobold API with the provided prompt and
generate text.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param endpoint: str [Required]¶
The API endpoint to use for generating text.
param max_context_length: Optional[int] = 1600¶
Maximum number of tokens to send to the model.
minimum: 1
param max_length: Optional[int] = 80¶
Number of tokens to generate.
maximum: 512
minimum: 1
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param rep_pen: Optional[float] = 1.12¶
Base repetition penalty value.
minimum: 1
param rep_pen_range: Optional[int] = 1024¶
Repetition penalty range.
minimum: 0
param rep_pen_slope: Optional[float] = 0.9¶
Repetition penalty slope.
minimum: 0
param tags: Optional[List[str]] = None¶
minimum: 0
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: Optional[float] = 0.6¶
Temperature value.
exclusiveMinimum: 0
param tfs: Optional[float] = 0.9¶
Tail free sampling value.
maximum: 1
minimum: 0
param top_a: Optional[float] = 0.9¶
Top-a sampling value.
minimum: 0
param top_k: Optional[int] = 0¶
Top-k sampling value.
minimum: 0
param top_p: Optional[float] = 0.95¶
Top-p sampling value.
maximum: 1
minimum: 0
param typical: Optional[float] = 0.5¶
Typical sampling value.
maximum: 1
minimum: 0
param use_authors_note: Optional[bool] = False¶
Whether to use the author’s note from the KoboldAI GUI when generating text.
This has no effect unless use_story is also enabled.
param use_memory: Optional[bool] = False¶
Whether to use the memory from the KoboldAI GUI when generating text.
param use_story: Optional[bool] = False¶
Whether or not to use the story from the KoboldAI GUI when generating text.
param use_world_info: Optional[bool] = False¶
Whether to use the world info from the KoboldAI GUI when generating text.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using KoboldApiLLM¶
KoboldAI API
langchain_community.llms.vertexai.completion_with_retry¶
langchain_community.llms.vertexai.completion_with_retry(llm: VertexAI, prompt: List[Union[str, 'Image']], stream: bool = False, is_gemini: bool = False, run_manager: Optional[CallbackManagerForLLMRun] = None, **kwargs: Any) → Any[source]¶
Use tenacity to retry the completion call.
langchain_community.llms.google_palm.GooglePalm¶
class langchain_community.llms.google_palm.GooglePalm[source]¶
Bases: BaseLLM, BaseModel
[Deprecated]  DEPRECATED: Use langchain_google_genai.GoogleGenerativeAI instead.
Google PaLM models.[Deprecated] DEPRECATED: Use langchain_google_genai.GoogleGenerativeAI instead.
Google PaLM models.
Notes
Deprecated since version 0.0.351: Use langchain_google_genai.GoogleGenerativeAI instead.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param google_api_key: Optional[SecretStr] = None¶
Constraints
type = string
writeOnly = True
format = password
param max_output_tokens: Optional[int] = None¶
Maximum number of tokens to include in a candidate. Must be greater than zero.
If unset, will default to 64.
param max_retries: int = 6¶
The maximum number of retries to make when generating.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_name: str = 'models/text-bison-001'¶
Model name to use.
param n: int = 1¶
Number of chat completions to generate for each prompt. Note that the API may
not return the full n completions if duplicates are generated.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.7¶
Run inference with this temperature. Must by in the closed interval
Run inference with this temperature. Must by in the closed interval
[0.0, 1.0].
param top_k: Optional[int] = None¶
Decode using top-k sampling: consider the set of top_k most probable tokens.
Must be positive.
param top_p: Optional[float] = None¶
Decode using nucleus sampling: consider the smallest set of tokens whose
probability sum is at least top_p. Must be in the closed interval [0.0, 1.0].
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
async abatch(inputs: List[Union[PromptValue, str, List[BaseMessage]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
added to the run.
with_retry(*, retry_if_exception_type: ~typing.Tuple[~typing.Type[BaseException], ...] = (<class 'Exception'>,), wait_exponential_jitter: bool = True, stop_after_attempt: int = 3) → Runnable[Input, Output]¶
Create a new Runnable that retries the original runnable on exceptions.
Parameters
retry_if_exception_type – A tuple of exception types to retry on
wait_exponential_jitter – Whether to add jitter to the wait time
between retries
stop_after_attempt – The maximum number of attempts to make before giving up
Returns
A new Runnable that retries the original runnable on exceptions.
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property is_gemini: bool¶
Returns whether a model is belongs to a Gemini family or not.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
langchain_community.llms.ollama.Ollama¶
class langchain_community.llms.ollama.Ollama[source]¶
Bases: BaseLLM, _OllamaCommon
Ollama locally runs large language models.
To use, follow the instructions at https://ollama.ai/.
Example
from langchain_community.llms import Ollama
ollama = Ollama(model="llama2")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param base_url: str = 'http://localhost:11434'¶
Base url the model is hosted under.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param format: Optional[str] = None¶
Specify the format of the output (e.g., json)
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param mirostat: Optional[int] = None¶
Enable Mirostat sampling for controlling perplexity.
(default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
param mirostat_eta: Optional[float] = None¶
Influences how quickly the algorithm responds to feedback
from the generated text. A lower learning rate will result in
slower adjustments, while a higher learning rate will make
the algorithm more responsive. (Default: 0.1)
param mirostat_tau: Optional[float] = None¶
Controls the balance between coherence and diversity
of the output. A lower value will result in more focused and
coherent text. (Default: 5.0)
coherent text. (Default: 5.0)
param model: str = 'llama2'¶
Model name to use.
param num_ctx: Optional[int] = None¶
Sets the size of the context window used to generate the
next token. (Default: 2048)
param num_gpu: Optional[int] = None¶
The number of GPUs to use. On macOS it defaults to 1 to
enable metal support, 0 to disable.
param num_thread: Optional[int] = None¶
Sets the number of threads to use during computation.
By default, Ollama will detect this for optimal performance.
It is recommended to set this value to the number of physical
CPU cores your system has (as opposed to the logical number of cores).
param repeat_last_n: Optional[int] = None¶
Sets how far back for the model to look back to prevent
repetition. (Default: 64, 0 = disabled, -1 = num_ctx)
param repeat_penalty: Optional[float] = None¶
Sets how strongly to penalize repetitions. A higher value (e.g., 1.5)
will penalize repetitions more strongly, while a lower value (e.g., 0.9)
will be more lenient. (Default: 1.1)
param stop: Optional[List[str]] = None¶
Sets the stop tokens to use.
param system: Optional[str] = None¶
system prompt (overrides what is defined in the Modelfile)
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: Optional[float] = None¶
The temperature of the model. Increasing the temperature will
make the model answer more creatively. (Default: 0.8)
param template: Optional[str] = None¶
param template: Optional[str] = None¶
full prompt or prompt template (overrides what is defined in the Modelfile)
param tfs_z: Optional[float] = None¶
Tail free sampling is used to reduce the impact of less probable
tokens from the output. A higher value (e.g., 2.0) will reduce the
impact more, while a value of 1.0 disables this setting. (default: 1)
param timeout: Optional[int] = None¶
Timeout for the request stream
param top_k: Optional[int] = None¶
Reduces the probability of generating nonsense. A higher value (e.g. 100)
will give more diverse answers, while a lower value (e.g. 10)
will be more conservative. (Default: 40)
param top_p: Optional[int] = None¶
Works together with top-k. A higher value (e.g., 0.95) will lead
to more diverse text, while a lower value (e.g., 0.5) will
generate more focused and conservative text. (Default: 0.9)
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
async abatch(inputs: List[Union[PromptValue, str, List[BaseMessage]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using Ollama¶
Ollama
Run LLMs locally