Package: llama-index-readers-opendal-reader
OpenDAL LoadersBase OpendalReaderThis loader parses any file viaApache OpenDAL.All files are temporarily downloaded locally and subsequently parsed withSimpleDirectoryReader. Hence, you may also specify a customfile_extractor, relying on any of the loaders in this library (or your own)!UsageOpendalReadercan read data from any supported storage services includings3,azblob,gcsand so on.fromllama_indeximportdownload_loaderOpendalReader=download_loader("OpendalReader")loader=OpendalReader(scheme="s3",bucket="bucket",path="path/to/data/",)documents=loader.load_data()We also provideOpendal[S3|Gcs|Azblob]Readerfor convenience.This loader is designed to be used as a way to load data intoLlamaIndexand/or subsequently used as a Tool in aLangChainAgent. Seeherefor examples.Azblob LoaderThis loader parses any file stored on Azblob.All files are temporarily downloaded locally and subsequently parsed withSimpleDirectoryReader. Hence, you may also specify a customfile_extractor, relying on any of the loaders in this library (or your own)!Azblob loader is based onOpendalReader.Usagefromllama_indeximportdownload_loaderOpendalAzblobReader=download_loader("OpendalAzblobReader")loader=OpendalAzblobReader(container="container",path="path/to/data/",endpoint="[endpoint]",account_name="[account_name]",account_key="[account_key]",)documents=loader.load_data()This loader is designed to be used as a way to load data intoLlamaIndexand/or subsequently used as a Tool in aLangChainAgent. Seeherefor examples.Gcs LoaderThis loader parses any file stored on Gcs.All files are temporarily downloaded locally and subsequently parsed withSimpleDirectoryReader. Hence, you may also specify a customfile_extractor, relying on any of the loaders in this library (or your own)!Gcs loader is based onOpendalReader.Usagefromllama_indeximportdownload_loaderOpendalGcsReader=download_loader("OpendalGcsReader")loader=OpendalGcsReader(bucket="bucket",path="path/to/data/",endpoint="[endpoint]",credentials="[credentials]",)documents=loader.load_data()Note: ifcredentialsis not provided, this loader to try to load from env.This loader is designed to be used as a way to load data intoLlamaIndexand/or subsequently used as a Tool in aLangChainAgent. Seeherefor examples.S3 LoaderThis loader parses any file stored on S3. When initializingS3Reader, you may pass in yourAWS Access Key. If none are found, the loader assumes they are stored in~/.aws/credentials.All files are temporarily downloaded locally and subsequently parsed withSimpleDirectoryReader. Hence, you may also specify a customfile_extractor, relying on any of the loaders in this library (or your own)!S3 loader is based onOpendalReader.Usagefromllama_indeximportdownload_loaderOpendalS3Reader=download_loader("OpendalS3Reader")loader=OpendalS3Reader(bucket="bucket",path="path/to/data/",access_key_id="[ACCESS_KEY_ID]",secret_access_key="[ACCESS_KEY_SECRET]",)documents=loader.load_data()Note: ifaccess_key_idorsecret_access_keyis not provided, this loader to try to load from env.Possible arguments includes:endpoint: Specify the endpoint of s3 service.region: Specify the region of s3 service.This loader is designed to be used as a way to load data intoLlamaIndexand/or subsequently used as a Tool in aLangChainAgent. Seeherefor examples.