Tony20100/langchain_full huggingface dataset:
langchain_community.document_loaders.oracleadb_loader.OracleAutonomousDatabaseLoader¶
class langchain_community.document_loaders.oracleadb_loader.OracleAutonomousDatabaseLoader(query: str, user: str, password: str, *, schema: Optional[str] = None, tns_name: Optional[str] = None, config_dir: Optional[str] = None, wallet_location: Optional[str] = None, wallet_password: Optional[str] = None, connection_string: Optional[str] = None, metadata: Optional[List[str]] = None)[source]¶
Load from oracle adb
Autonomous Database connection can be made by either connection_string
or tns name. wallet_location and wallet_password are required
for TLS connection.
Each document will represent one row of the query result.
Columns are written into the page_content and ‘metadata’ in
constructor is written into ‘metadata’ of document,
by default, the ‘metadata’ is None.
init method
:param query: sql query to execute
:param user: username
:param password: user password
:param schema: schema to run in database
:param tns_name: tns name in tnsname.ora
:param config_dir: directory of config files(tnsname.ora, wallet)
:param wallet_location: location of wallet
:param wallet_password: password of wallet
:param connection_string: connection string to connect to adb instance
:param metadata: metadata used in document
Methods
__init__(query, user, password, *[, schema, ...])
init method :param query: sql query to execute :param user: username :param password: user password :param schema: schema to run in database :param tns_name: tns name in tnsname.ora :param config_dir: directory of config files(tnsname.ora, wallet) :param wallet_location: location of wallet :param wallet_password: password of wallet :param connection_string: connection string to connect to adb instance :param metadata: metadata used in document
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
A lazy loader for Documents.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
Parameters
query (str) – 
user (str) – 
password (str) – 
schema (Optional[str]) – 
tns_name (Optional[str]) – 
config_dir (Optional[str]) – 
wallet_location (Optional[str]) – 
wallet_password (Optional[str]) – 
connection_string (Optional[str]) – 
metadata (Optional[List[str]]) – 
__init__(query: str, user: str, password: str, *, schema: Optional[str] = None, tns_name: Optional[str] = None, config_dir: Optional[str] = None, wallet_location: Optional[str] = None, wallet_password: Optional[str] = None, connection_string: Optional[str] = None, metadata: Optional[List[str]] = None)[source]¶
init method
:param query: sql query to execute
:param user: username
:param password: user password
:param schema: schema to run in database
:param tns_name: tns name in tnsname.ora
:param config_dir: directory of config files(tnsname.ora, wallet)
:param wallet_location: location of wallet
:param wallet_location: location of wallet
:param wallet_password: password of wallet
:param connection_string: connection string to connect to adb instance
:param metadata: metadata used in document
Parameters
query (str) – 
user (str) – 
password (str) – 
schema (Optional[str]) – 
tns_name (Optional[str]) – 
config_dir (Optional[str]) – 
wallet_location (Optional[str]) – 
wallet_password (Optional[str]) – 
connection_string (Optional[str]) – 
metadata (Optional[List[str]]) – 
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
A lazy loader for Documents.
Return type
Iterator[Document]
load() → List[Document][source]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using OracleAutonomousDatabaseLoader¶
Oracle Autonomous Database
langchain_community.document_loaders.geodataframe.GeoDataFrameLoader¶
class langchain_community.document_loaders.geodataframe.GeoDataFrameLoader(data_frame: Any, page_content_column: str = 'geometry')[source]¶
Load geopandas Dataframe.
Initialize with geopandas Dataframe.
Parameters
data_frame (Any) – geopandas DataFrame object.
page_content_column (str) – Name of the column containing the page content.
Defaults to “geometry”.
Methods
__init__(data_frame[, page_content_column])
Initialize with geopandas Dataframe.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
Lazy load records from dataframe.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
__init__(data_frame: Any, page_content_column: str = 'geometry')[source]¶
Initialize with geopandas Dataframe.
Parameters
data_frame (Any) – geopandas DataFrame object.
page_content_column (str) – Name of the column containing the page content.
Defaults to “geometry”.
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document][source]¶
Lazy load records from dataframe.
Return type
Iterator[Document]
load() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using GeoDataFrameLoader¶
Geopandas
langchain_community.document_loaders.parsers.language.lua.LuaSegmenter¶
class langchain_community.document_loaders.parsers.language.lua.LuaSegmenter(code: str)[source]¶
Code segmenter for Lua.
Methods
__init__(code)
extract_functions_classes()
get_chunk_query()
get_language()
get_parser()
is_valid()
make_line_comment(text)
simplify_code()
Parameters
code (str) – 
__init__(code: str)¶
Parameters
code (str) – 
extract_functions_classes() → List[str]¶
Return type
List[str]
get_chunk_query() → str[source]¶
Return type
str
get_language() → Language[source]¶
Return type
Language
get_parser() → Parser¶
Return type
Parser
is_valid() → bool¶
Return type
bool
make_line_comment(text: str) → str[source]¶
Parameters
text (str) – 
Return type
str
simplify_code() → str¶
Return type
str
langchain_community.document_loaders.lakefs.UnstructuredLakeFSLoader¶
class langchain_community.document_loaders.lakefs.UnstructuredLakeFSLoader(url: str, repo: str, ref: str = 'main', path: str = '', presign: bool = True, **unstructured_kwargs: Any)[source]¶
Load from lakeFS as unstructured data.
Initialize UnstructuredLakeFSLoader.
Args:
Parameters
lakefs_access_key – 
lakefs_secret_key – 
lakefs_endpoint – 
repo (str) – 
ref (str) – 
url (str) – 
path (str) – 
presign (bool) – 
unstructured_kwargs (Any) – 
Methods
__init__(url, repo[, ref, path, presign])
Initialize UnstructuredLakeFSLoader.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
Load file.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
__init__(url: str, repo: str, ref: str = 'main', path: str = '', presign: bool = True, **unstructured_kwargs: Any)[source]¶
Initialize UnstructuredLakeFSLoader.
Args:
Parameters
lakefs_access_key – 
lakefs_secret_key – 
lakefs_endpoint – 
repo (str) – 
ref (str) – 
url (str) – 
path (str) – 
presign (bool) – 
unstructured_kwargs (Any) – 
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
Load file.
Return type
Iterator[Document]
load() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
langchain_community.document_loaders.surrealdb.SurrealDBLoader¶
class langchain_community.document_loaders.surrealdb.SurrealDBLoader(filter_criteria: Optional[Dict] = None, **kwargs: Any)[source]¶
Load SurrealDB documents.
Methods
__init__([filter_criteria])
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
initialize()
Initialize connection to surrealdb database and authenticate if credentials are provided
lazy_load()
A lazy loader for Documents.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
Parameters
filter_criteria (Optional[Dict]) – 
kwargs (Any) – 
__init__(filter_criteria: Optional[Dict] = None, **kwargs: Any) → None[source]¶
Parameters
filter_criteria (Optional[Dict]) – 
kwargs (Any) – 
Return type
None
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document][source]¶
Load data into Document objects.
Return type
List[Document]
async initialize() → None[source]¶
Initialize connection to surrealdb database
and authenticate if credentials are provided
Return type
None
lazy_load() → Iterator[Document]¶
A lazy loader for Documents.
Return type
Iterator[Document]
load() → List[Document][source]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
langchain_community.document_loaders.confluence.ContentFormat¶
class langchain_community.document_loaders.confluence.ContentFormat(value)[source]¶
Enumerator of the content formats of Confluence page.
EDITOR = 'body.editor'¶
EXPORT_VIEW = 'body.export_view'¶
ANONYMOUS_EXPORT_VIEW = 'body.anonymous_export_view'¶
STORAGE = 'body.storage'¶
VIEW = 'body.view'¶
langchain_community.document_loaders.url.UnstructuredURLLoader¶
class langchain_community.document_loaders.url.UnstructuredURLLoader(urls: List[str], continue_on_failure: bool = True, mode: str = 'single', show_progress_bar: bool = False, **unstructured_kwargs: Any)[source]¶
Load files from remote URLs using Unstructured.
Use the unstructured partition function to detect the MIME type
and route the file to the appropriate partitioner.
You can run the loader in one of two modes: “single” and “elements”.
If you use “single” mode, the document will be returned as a single
langchain Document object. If you use “elements” mode, the unstructured
library will split the document into elements such as Title and NarrativeText.
You can pass in additional unstructured kwargs after mode to apply
different unstructured settings.
Examples
from langchain_community.document_loaders import UnstructuredURLLoader
loader = UnstructuredURLLoader(urls=[“<url-1>”, “<url-2>”], mode=”elements”, strategy=”fast”,
)
docs = loader.load()
References
https://unstructured-io.github.io/unstructured/bricks.html#partition
Initialize with file path.
Methods
__init__(urls[, continue_on_failure, mode, ...])
Initialize with file path.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
A lazy loader for Documents.
load()
Load file.
load_and_split([text_splitter])
Load Documents and split into chunks.
Parameters
urls (List[str]) – 
continue_on_failure (bool) – 
mode (str) – 
show_progress_bar (bool) – 
unstructured_kwargs (Any) –
show_progress_bar (bool) – 
unstructured_kwargs (Any) – 
__init__(urls: List[str], continue_on_failure: bool = True, mode: str = 'single', show_progress_bar: bool = False, **unstructured_kwargs: Any)[source]¶
Initialize with file path.
Parameters
urls (List[str]) – 
continue_on_failure (bool) – 
mode (str) – 
show_progress_bar (bool) – 
unstructured_kwargs (Any) – 
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
A lazy loader for Documents.
Return type
Iterator[Document]
load() → List[Document][source]¶
Load file.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using UnstructuredURLLoader¶
URL
Unstructured
langchain_community.document_loaders.tencent_cos_file.TencentCOSFileLoader¶
class langchain_community.document_loaders.tencent_cos_file.TencentCOSFileLoader(conf: Any, bucket: str, key: str)[source]¶
Load from Tencent Cloud COS file.
Initialize with COS config, bucket and key name.
:param conf(CosConfig): COS config.
:param bucket(str): COS bucket.
:param key(str): COS file key.
Methods
__init__(conf, bucket, key)
Initialize with COS config, bucket and key name.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
Load documents.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
Parameters
conf (Any) – 
bucket (str) – 
key (str) – 
__init__(conf: Any, bucket: str, key: str)[source]¶
Initialize with COS config, bucket and key name.
:param conf(CosConfig): COS config.
:param bucket(str): COS bucket.
:param key(str): COS file key.
Parameters
conf (Any) – 
bucket (str) – 
key (str) – 
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document][source]¶
Load documents.
Return type
Iterator[Document]
load() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using TencentCOSFileLoader¶
Tencent
Tencent COS File
langchain_community.document_loaders.pdf.UnstructuredPDFLoader¶
class langchain_community.document_loaders.pdf.UnstructuredPDFLoader(file_path: Optional[Union[str, List[str], Path, List[Path]]], mode: str = 'single', **unstructured_kwargs: Any)[source]¶
Load PDF files using Unstructured.
You can run the loader in one of two modes: “single” and “elements”.
If you use “single” mode, the document will be returned as a single
langchain Document object. If you use “elements” mode, the unstructured
library will split the document into elements such as Title and NarrativeText.
You can pass in additional unstructured kwargs after mode to apply
different unstructured settings.
Examples
from langchain_community.document_loaders import UnstructuredPDFLoader
loader = UnstructuredPDFLoader(“example.pdf”, mode=”elements”, strategy=”fast”,
)
docs = loader.load()
References
https://unstructured-io.github.io/unstructured/bricks.html#partition-pdf
Initialize with file path.
Methods
__init__(file_path[, mode])
Initialize with file path.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
Load file.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
Parameters
file_path (Optional[Union[str, List[str], Path, List[Path]]]) – 
mode (str) – 
unstructured_kwargs (Any) – 
__init__(file_path: Optional[Union[str, List[str], Path, List[Path]]], mode: str = 'single', **unstructured_kwargs: Any)¶
Initialize with file path.
Parameters
Initialize with file path.
Parameters
file_path (Optional[Union[str, List[str], Path, List[Path]]]) – 
mode (str) – 
unstructured_kwargs (Any) – 
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
Load file.
Return type
Iterator[Document]
load() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using UnstructuredPDFLoader¶
How to load PDFs
Unstructured
langchain_community.document_loaders.parsers.language.python.PythonSegmenter¶
class langchain_community.document_loaders.parsers.language.python.PythonSegmenter(code: str)[source]¶
Code segmenter for Python.
Methods
__init__(code)
extract_functions_classes()
is_valid()
simplify_code()
Parameters
code (str) – 
__init__(code: str)[source]¶
Parameters
code (str) – 
extract_functions_classes() → List[str][source]¶
Return type
List[str]
is_valid() → bool[source]¶
Return type
bool
simplify_code() → str[source]¶
Return type
str
langchain_community.document_loaders.unstructured.UnstructuredFileLoader¶
class langchain_community.document_loaders.unstructured.UnstructuredFileLoader(file_path: Optional[Union[str, List[str], Path, List[Path]]], mode: str = 'single', **unstructured_kwargs: Any)[source]¶
Load files using Unstructured.
The file loader uses the
unstructured partition function and will automatically detect the file
type. You can run the loader in one of two modes: “single” and “elements”.
If you use “single” mode, the document will be returned as a single
langchain Document object. If you use “elements” mode, the unstructured
library will split the document into elements such as Title and NarrativeText.
You can pass in additional unstructured kwargs after mode to apply
different unstructured settings.
Examples
from langchain_community.document_loaders import UnstructuredFileLoader
loader = UnstructuredFileLoader(“example.pdf”, mode=”elements”, strategy=”fast”,
)
docs = loader.load()
References
https://unstructured-io.github.io/unstructured/bricks.html#partition
Initialize with file path.
Methods
__init__(file_path[, mode])
Initialize with file path.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
Load file.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
Parameters
file_path (Optional[Union[str, List[str], Path, List[Path]]]) – 
mode (str) – 
unstructured_kwargs (Any) – 
__init__(file_path: Optional[Union[str, List[str], Path, List[Path]]], mode: str = 'single', **unstructured_kwargs: Any)[source]¶
Initialize with file path.
Parameters
file_path (Optional[Union[str, List[str], Path, List[Path]]]) – 
mode (str) – 
unstructured_kwargs (Any) – 
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
Load file.
Return type
Iterator[Document]
load() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents.
Defaults to RecursiveCharacterTextSplitter.
Returns
List of Documents.
Return type
List[Document]
Examples using UnstructuredFileLoader¶
Unstructured
Unstructured File
langchain_community.document_loaders.base_o365.fetch_mime_types¶
langchain_community.document_loaders.base_o365.fetch_mime_types(file_types: Sequence[_FileType]) → Dict[str, str][source]¶
Fetch the mime types for the specified file types.
Parameters
file_types (Sequence[_FileType]) – 
Return type
Dict[str, str]
langchain_community.document_loaders.parsers.language.code_segmenter.CodeSegmenter¶
class langchain_community.document_loaders.parsers.language.code_segmenter.CodeSegmenter(code: str)[source]¶
Abstract class for the code segmenter.
Methods
__init__(code)
extract_functions_classes()
is_valid()
simplify_code()
Parameters
code (str) – 
__init__(code: str)[source]¶
Parameters
code (str) – 
abstract extract_functions_classes() → List[str][source]¶
Return type
List[str]
is_valid() → bool[source]¶
Return type
bool
abstract simplify_code() → str[source]¶
Return type
str
langchain_community.document_loaders.iugu.IuguLoader¶
class langchain_community.document_loaders.iugu.IuguLoader(resource: str, api_token: Optional[str] = None)[source]¶
Load from IUGU.
Initialize the IUGU resource.
Parameters
resource (str) – The name of the resource to fetch.
api_token (Optional[str]) – The IUGU API token to use.
Methods
__init__(resource[, api_token])
Initialize the IUGU resource.
alazy_load()
A lazy loader for Documents.
aload()
Load data into Document objects.
lazy_load()
A lazy loader for Documents.
load()
Load data into Document objects.
load_and_split([text_splitter])
Load Documents and split into chunks.
__init__(resource: str, api_token: Optional[str] = None) → None[source]¶
Initialize the IUGU resource.
Parameters
resource (str) – The name of the resource to fetch.
api_token (Optional[str]) – The IUGU API token to use.
Return type
None
async alazy_load() → AsyncIterator[Document]¶
A lazy loader for Documents.
Return type
AsyncIterator[Document]
async aload() → List[Document]¶
Load data into Document objects.
Return type
List[Document]
lazy_load() → Iterator[Document]¶
A lazy loader for Documents.
Return type
Iterator[Document]
load() → List[Document][source]¶
Load data into Document objects.
Return type
List[Document]
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
Parameters