Repository: StarRocks/starrocks
Pull request #59061: [Enhancement] Add azure blob file system in backend
Author: login: wyb
Labels: name: behavior_changed
name: 3.5-merged
Created: 2025-05-19T02:16:26Z | Merged: 2025-05-22T02:06:59Z

## Why I'm doing:

## What I'm doing:

1. Add a new `AzBlobFileSystem` for creating read and write files, and we can implement more functions later.
2. Add `AzBlobInputStream` for reading azure blob.
3. Add `AzBlobOutputStream` for writing azure blob.
4. Support using managed identity to read and write azure blob.

Fixes #59017

## What type of PR is this:

- [ ] BugFix
- [ ] Feature
- [x] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool

Does this PR entail a change in behavior?

- [x] Yes, this PR will result in a change in behavior.
- [ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [x] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

## Checklist:

- [x] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
  - [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr

## Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
  - [x] 3.5
  - [ ] 3.4
  - [ ] 3.3
  - [ ] 3.2
  - [ ] 3.1


Linked issue #59017: Read and write azure blob using azure native cpp sdk in files() table function
## Enhancement

Currently, starrocks uses `hadoop azure` to read and write `azure blob`, but `hadoop azure` does not support `managed identity` to access `azure blob`.

So i want to add `azure native sdk` for reading and writing `azure blob` using `managed identity`.


```
// user assigned managed identity
SELECT * FROM FILES(
    "path" = "wasbs://container@account.blob.core.windows.net/path/*",
    "format" = "parquet",
    "azure.blob.oauth2_use_managed_identity" = "true",
    "azure.blob.oauth2_client_id" = "xxx"
)

INSERT INTO FILES(
    "path" = "wasbs://container@account.blob.core.windows.net/path/",
    "format" = "parquet",
    "azure.blob.oauth2_use_managed_identity" = "true",
    "azure.blob.oauth2_client_id" = "xxx"
) SELECT * FROM SR;

// client secret service principal
SELECT * FROM FILES(
    "path" = "wasbs://container@account.blob.core.windows.net/path/*",
    "format" = "parquet",
    "azure.blob.oauth2_client_id" = "xxx",
    "azure.blob.oauth2_client_secret" = "xxx",
    "azure.blob.oauth2_tenant_id" = "xxx"
)
```

PRs:
- [x] https://github.com/StarRocks/starrocks/pull/59016
- [x] https://github.com/StarRocks/starrocks/pull/59062
- [x] https://github.com/StarRocks/starrocks/pull/59066
- [x] https://github.com/StarRocks/starrocks/pull/59061
- [x] https://github.com/StarRocks/starrocks/pull/59097
- [x] https://github.com/StarRocks/starrocks/pull/59160
- [x] https://github.com/StarRocks/starrocks/pull/59059
- [x] https://github.com/StarRocks/starrocks/pull/59308
- [x] https://github.com/StarRocks/starrocks/pull/59657


Linked issue comments: user: login: jaogoy
created_at: 2025-05-20T05:48:58Z
body: @wyb You can list all the PRs in main for a clear view.
author_association: CONTRIBUTOR

Diff:
diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
index 1ccece030906e6..b545672a24635f 100644
--- a/be/CMakeLists.txt
+++ b/be/CMakeLists.txt
@@ -407,6 +407,24 @@ set_target_properties(serdes PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/
 add_library(opentelemetry_exporter_jaeger_trace STATIC IMPORTED GLOBAL)
 set_target_properties(opentelemetry_exporter_jaeger_trace PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib64/libopentelemetry_exporter_jaeger_trace.a)
 
+add_library(libxml2 STATIC IMPORTED GLOBAL)
+set_target_properties(libxml2 PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/libxml2.a)
+
+add_library(azure-core STATIC IMPORTED GLOBAL)
+set_target_properties(azure-core PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/libazure-core.a)
+
+add_library(azure-identity STATIC IMPORTED GLOBAL)
+set_target_properties(azure-identity PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/libazure-identity.a)
+
+add_library(azure-storage-common STATIC IMPORTED GLOBAL)
+set_target_properties(azure-storage-common PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/libazure-storage-common.a)
+
+add_library(azure-storage-blobs STATIC IMPORTED GLOBAL)
+set_target_properties(azure-storage-blobs PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/libazure-storage-blobs.a)
+
+add_library(azure-storage-files-datalake STATIC IMPORTED GLOBAL)
+set_target_properties(azure-storage-files-datalake PROPERTIES IMPORTED_LOCATION ${THIRDPARTY_DIR}/lib/libazure-storage-files-datalake.a)
+
 set(absl_DIR "${THIRDPARTY_DIR}/lib/cmake/absl" CACHE PATH "absl search path" FORCE)
 find_package(absl CONFIG REQUIRED)
 set(gRPC_DIR "${THIRDPARTY_DIR}/lib/cmake/grpc" CACHE PATH "grpc search path")
@@ -469,12 +487,12 @@ if ("${USE_STAROS}" STREQUAL "ON")
     add_library(rocksdb::rocksdb ALIAS rocksdb)
     add_library(starcache::starcache ALIAS starcache)
     add_library(Snappy::snappy ALIAS snappy)
-
-    # required by azure-*
-    set(LIBXML2_INCLUDE_DIR "${STARLET_INSTALL_DIR}/third_party/include/libxml2" CACHE PATH "libxml2 include path")
-    set(LIBXML2_LIBRARY "${STARLET_INSTALL_DIR}/third_party/lib/libxml2.a" CACHE FILEPATH "libxml2 static library")
-    add_library(libxml2 STATIC IMPORTED GLOBAL)
-    set_target_properties(libxml2 PROPERTIES IMPORTED_LOCATION ${LIBXML2_LIBRARY})
+    add_library(libxml2::libxml2 ALIAS libxml2)
+    add_library(Azure::azure-core ALIAS azure-core)
+    add_library(Azure::azure-identity ALIAS azure-identity)
+    add_library(Azure::azure-storage-common ALIAS azure-storage-common)
+    add_library(Azure::azure-storage-blobs ALIAS azure-storage-blobs)
+    add_library(Azure::azure-storage-files-datalake ALIAS azure-storage-files-datalake)
 
     set(prometheus-cpp_DIR "${STARLET_INSTALL_DIR}/third_party/lib/cmake/prometheus-cpp" CACHE PATH "prometheus cpp client search path")
     find_package (prometheus-cpp CONFIG REQUIRED)
@@ -483,18 +501,6 @@ if ("${USE_STAROS}" STREQUAL "ON")
     message(STATUS "  include: ${prometheus-cpp_INCLUDE_DIR}")
     include_directories(SYSTEM ${prometheus-cpp_INCLUDE_DIR})
 
-    set(azure-core-cpp_DIR "${STARLET_INSTALL_DIR}/third_party/share/azure-core-cpp" CACHE PATH "azure core cpp search path")
-    find_package(azure-core-cpp CONFIG REQUIRED)
-    set(azure-identity-cpp_DIR "${STARLET_INSTALL_DIR}/third_party/share/azure-identity-cpp" CACHE PATH "azure identity cpp search path")
-    find_package(azure-identity-cpp CONFIG REQUIRED)
-    set(azure-storage-common-cpp_DIR "${STARLET_INSTALL_DIR}/third_party/share/azure-storage-common-cpp" CACHE PATH "azure storage common search path")
-    find_package(azure-storage-common-cpp CONFIG REQUIRED)
-    set(azure-storage-blobs-cpp_DIR "${STARLET_INSTALL_DIR}/third_party/share/azure-storage-blobs-cpp" CACHE PATH "azure storage blobs search path")
-    find_package(azure-storage-blobs-cpp CONFIG REQUIRED)
-    set(azure-storage-files-datalake-cpp_DIR "${STARLET_INSTALL_DIR}/third_party/share/azure-storage-files-datalake-cpp" CACHE PATH "azure storage file datalake search path")
-    find_package(azure-storage-files-datalake-cpp CONFIG REQUIRED)
-    set(AZURE_SDK_LIB Azure::azure-identity Azure::azure-storage-blobs Azure::azure-storage-files-datalake Azure::azure-core Azure::azure-storage-common)
-
     # gcp cloud storage related dependencies
     set(ZLIB_INCLUDE_DIR "${THIRDPARTY_DIR}/include")
     set(ZLIB_LIBRARY "${THIRDPARTY_DIR}/lib/libz.a")
@@ -963,6 +969,12 @@ set(STARROCKS_DEPENDENCIES
     opentelemetry_resources
     opentelemetry_common
     opentelemetry_exporter_jaeger_trace
+    libxml2
+    azure-core
+    azure-identity
+    azure-storage-common
+    azure-storage-blobs
+    azure-storage-files-datalake
     jansson
     avro
     avrocpp
diff --git a/be/src/common/status.h b/be/src/common/status.h
index 7118887d0a9588..8d4155b1758fd8 100644
--- a/be/src/common/status.h
+++ b/be/src/common/status.h
@@ -162,9 +162,12 @@ class STATUS_ATTRIBUTE Status {
     static Status BigQueryScanRowsLimitExceeded(std::string_view msg) {
         return Status(TStatusCode::BIG_QUERY_SCAN_ROWS_LIMIT_EXCEEDED, msg);
     }
+    static Status NotAuthorized(std::string_view msg) { return Status(TStatusCode::NOT_AUTHORIZED, msg); }
 
     bool ok() const { return _state == nullptr; }
 
+    bool is_unknown() const { return code() == TStatusCode::UNKNOWN; }
+
     bool is_cancelled() const { return code() == TStatusCode::CANCELLED; }
 
     bool is_mem_limit_exceeded() const { return code() == TStatusCode::MEM_LIMIT_EXCEEDED; }
@@ -219,6 +222,8 @@ class STATUS_ATTRIBUTE Status {
 
     bool is_shutdown() const { return code() == TStatusCode::SHUTDOWN; }
 
+    bool is_not_authorized() const { return code() == TStatusCode::NOT_AUTHORIZED; }
+
     // Convert into TStatus. Call this if 'status_container' contains an optional
     // TStatus field named 'status'. This also sets __isset.status.
     template <typename T>
diff --git a/be/src/fs/CMakeLists.txt b/be/src/fs/CMakeLists.txt
index 50f20d0dc4c16b..c03323f963123c 100644
--- a/be/src/fs/CMakeLists.txt
+++ b/be/src/fs/CMakeLists.txt
@@ -27,6 +27,8 @@ set(EXEC_FILES
     fs_memory.cpp
     fs_s3.cpp
     azure/azblob_uri.cpp
+    azure/fs_azblob.cpp
+    azure/utils.cpp
     credential/cloud_configuration_factory.cpp
     hdfs/fs_hdfs.cpp
     hdfs/hdfs_fs_cache.cpp
diff --git a/be/src/fs/azure/fs_azblob.cpp b/be/src/fs/azure/fs_azblob.cpp
new file mode 100644
index 00000000000000..a3d25b0304bc29
--- /dev/null
+++ b/be/src/fs/azure/fs_azblob.cpp
@@ -0,0 +1,540 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fs/azure/fs_azblob.h"
+
+#include <fmt/format.h>
+
+#include <azure/identity.hpp>
+#include <azure/storage/blobs.hpp>
+
+#include "fs/azure/azblob_uri.h"
+#include "fs/azure/utils.h"
+#include "fs/credential/cloud_configuration_factory.h"
+#include "fs/encrypt_file.h"
+#include "fs/output_stream_adapter.h"
+#include "io/io_profiler.h"
+#include "io/output_stream.h"
+#include "util/stopwatch.hpp"
+
+namespace starrocks {
+
+using BlobContainerClient = Azure::Storage::Blobs::BlobContainerClient;
+using BlobContainerClientPtr = std::shared_ptr<BlobContainerClient>;
+using BlockBlobClient = Azure::Storage::Blobs::BlockBlobClient;
+using BlockBlobClientUniquePtr = std::unique_ptr<Azure::Storage::Blobs::BlockBlobClient>;
+
+// ------ input stream ------
+
+class AzBlobInputStream : public io::SeekableInputStream {
+public:
+    explicit AzBlobInputStream(BlockBlobClientUniquePtr client, AzBlobURI uri)
+            : _blob_client(std::move(client)), _uri(std::move(uri)) {}
+    ~AzBlobInputStream() override = default;
+
+    AzBlobInputStream(const AzBlobInputStream&) = delete;
+    void operator=(const AzBlobInputStream&) = delete;
+    AzBlobInputStream(AzBlobInputStream&&) = delete;
+    void operator=(AzBlobInputStream&&) = delete;
+
+    StatusOr<int64_t> read(void* data, int64_t size) override;
+    StatusOr<int64_t> get_size() override;
+    StatusOr<int64_t> position() override { return _offset; }
+    Status seek(int64_t offset) override;
+    void set_size(int64_t size) override { _size = size; }
+
+private:
+    BlockBlobClientUniquePtr _blob_client;
+    AzBlobURI _uri;
+    int64_t _offset{0};
+    int64_t _size{-1};
+};
+
+StatusOr<int64_t> AzBlobInputStream::read(void* data, int64_t count) {
+    if (UNLIKELY(_size == -1)) {
+        ASSIGN_OR_RETURN(_size, get_size());
+    }
+
+    if (_offset >= _size) { // 0 means reach EOF
+        return 0;
+    }
+
+    MonotonicStopWatch watch;
+    watch.start();
+
+    Azure::Core::Http::HttpRange range;
+    range.Offset = _offset;
+    range.Length = std::min<int64_t>(count, _size - _offset);
+
+    Azure::Storage::Blobs::DownloadBlobToOptions options;
+    options.Range = range;
+
+    int64_t size = 0;
+    try {
+        auto response = _blob_client->DownloadTo(reinterpret_cast<uint8_t*>(data), count, options);
+        if (!response.Value.ContentRange.Length.HasValue()) {
+            return Status::InternalError(fmt::format("Download blob object %s error: read length in response is empty",
+                                                     _uri.get_blob_uri()));
+        }
+        size = response.Value.ContentRange.Length.Value();
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(e.StatusCode,
+                                     fmt::format("Download blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                                     _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(fmt::format("Download blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    _offset += size;
+    IOProfiler::add_read(size, watch.elapsed_time());
+    return size;
+}
+
+StatusOr<int64_t> AzBlobInputStream::get_size() {
+    if (_size == -1) {
+        try {
+            auto response = _blob_client->GetProperties();
+            _size = response.Value.BlobSize;
+        } catch (const Azure::Core::RequestFailedException& e) {
+            return azure_error_to_status(
+                    e.StatusCode, fmt::format("Blob object {} get properties error: {}", _uri.get_blob_uri(), e.what()),
+                    _uri.get_blob_uri());
+        } catch (const std::exception& e) {
+            return Status::InternalError(
+                    fmt::format("Blob object {} get properties error: {}", _uri.get_blob_uri(), e.what()));
+        }
+    }
+    return _size;
+}
+
+Status AzBlobInputStream::seek(int64_t offset) {
+    if (offset < 0) {
+        return Status::InvalidArgument(fmt::format("Invalid offset {}", offset));
+    }
+
+    _offset = offset;
+    return Status::OK();
+}
+
+// ------ output stream ------
+
+class AzBlobOutputStream : public io::OutputStream {
+public:
+    explicit AzBlobOutputStream(BlockBlobClientUniquePtr client, AzBlobURI uri)
+            : _blob_client(std::move(client)), _uri(std::move(uri)) {}
+    ~AzBlobOutputStream() override = default;
+
+    AzBlobOutputStream(const AzBlobOutputStream&) = delete;
+    void operator=(const AzBlobOutputStream&) = delete;
+    AzBlobOutputStream(AzBlobOutputStream&&) = delete;
+    void operator=(AzBlobOutputStream&&) = delete;
+
+    [[nodiscard]] bool allows_aliasing() const override { return false; }
+    Status write_aliased(const void* data, int64_t size) override {
+        return Status::NotSupported("AzBlobOutputStream::write_aliased");
+    }
+    Status skip(int64_t count) override { return Status::NotSupported("AzBlobOutputStream::skip"); }
+    StatusOr<Buffer> get_direct_buffer() override {
+        return Status::NotSupported("AzBlobOutputStream::get_direct_buffer");
+    }
+    StatusOr<Position> get_direct_buffer_and_advance(int64_t size) override {
+        return Status::NotSupported("AzBlobOutputStream::get_direct_buffer_and_advance");
+    }
+
+    Status write(const void* data, int64_t size) override;
+    Status close() override;
+
+private:
+    Status create_multipart_upload();
+    Status multipart_upload();
+    Status singlepart_upload();
+    Status complete_multipart_upload();
+
+    // Max single object size stored in blob. For object size exceeds
+    // _max_single_part_size, multiupload will be used.
+    uint64_t _max_single_part_size = 100UL << 20;
+
+    // Upload part size if multiupload enabled.
+    uint64_t _min_upload_part_size = 5UL << 20;
+
+    BlockBlobClientUniquePtr _blob_client;
+    AzBlobURI _uri;
+
+    bool _multipart_upload{false};
+    std::string _buffer;
+    std::vector<std::string> _block_ids;
+};
+
+Status AzBlobOutputStream::write(const void* data, int64_t size) {
+    MonotonicStopWatch watch;
+    watch.start();
+
+    _buffer.append(static_cast<const char*>(data), size);
+    if (!_multipart_upload && _buffer.size() > _max_single_part_size) {
+        RETURN_IF_ERROR(create_multipart_upload());
+    }
+    if (_multipart_upload && _buffer.size() >= _min_upload_part_size) {
+        RETURN_IF_ERROR(multipart_upload());
+        _buffer.clear();
+    }
+
+    IOProfiler::add_write(size, watch.elapsed_time());
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::close() {
+    if (!_multipart_upload) {
+        RETURN_IF_ERROR(singlepart_upload());
+    } else {
+        RETURN_IF_ERROR(multipart_upload());
+        RETURN_IF_ERROR(complete_multipart_upload());
+    }
+
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::create_multipart_upload() {
+    _multipart_upload = true;
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::singlepart_upload() {
+    try {
+        _blob_client->UploadFrom(reinterpret_cast<const uint8_t*>(_buffer.data()), _buffer.size());
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(e.StatusCode,
+                                     fmt::format("Upload blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                                     _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(fmt::format("Upload blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    return Status::OK();
+}
+
+static std::string get_block_id(int64_t id) {
+    constexpr size_t kBlockIdLength = 64;
+    std::string block_id = std::to_string(id);
+    block_id = std::string(kBlockIdLength - block_id.length(), '0') + block_id;
+    return Azure::Core::Convert::Base64Encode(std::vector<uint8_t>(block_id.begin(), block_id.end()));
+}
+
+Status AzBlobOutputStream::multipart_upload() {
+    if (_buffer.empty()) {
+        return Status::OK();
+    }
+
+    try {
+        std::string block_id = get_block_id(_block_ids.size());
+        Azure::Core::IO::MemoryBodyStream body_stream(reinterpret_cast<const uint8_t*>(_buffer.data()), _buffer.size());
+        _blob_client->StageBlock(block_id, body_stream);
+        _block_ids.push_back(block_id);
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(e.StatusCode,
+                                     fmt::format("Stage blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                                     _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(fmt::format("Stage blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::complete_multipart_upload() {
+    try {
+        _blob_client->CommitBlockList(_block_ids);
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(
+                e.StatusCode,
+                fmt::format("Commit block list for blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(
+                fmt::format("Commit block list for blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    return Status::OK();
+}
+
+// ------ client factory ------
+
+class AzBlobClientFactory {
+public:
+    AzBlobClientFactory() { srand(time(nullptr)); }
+    ~AzBlobClientFactory() = default;
+
+    BlobContainerClientPtr new_blob_container_client(const AzureCloudCredential& azure_cloud_credential,
+                                                     const AzBlobURI& blob_uri);
+
+private:
+    BlobContainerClientPtr create_blob_container_client(const AzureCloudCredential& azure_cloud_credential,
+                                                        const AzBlobURI& blob_uri);
+
+    constexpr static int kMaxItems = 8;
+
+    struct ClientCacheItem {
+        AzureCloudCredential azure_cloud_credential;
+        BlobContainerClientPtr blob_container_client;
+    };
+
+    std::shared_mutex _lock;
+    std::vector<ClientCacheItem> _client_cache;
+};
+
+BlobContainerClientPtr AzBlobClientFactory::new_blob_container_client(
+        const AzureCloudCredential& azure_cloud_credential, const AzBlobURI& blob_uri) {
+    std::string container_uri = blob_uri.get_container_uri();
+    {
+        std::shared_lock guard(_lock);
+        for (auto& client : _client_cache) {
+            if (container_uri == client.blob_container_client->GetUrl() &&
+                azure_cloud_credential == client.azure_cloud_credential) {
+                return client.blob_container_client;
+            }
+        }
+    }
+
+    auto container_client = create_blob_container_client(azure_cloud_credential, blob_uri);
+
+    {
+        std::unique_lock guard(_lock);
+
+        for (auto& client : _client_cache) {
+            if (container_uri == client.blob_container_client->GetUrl() &&
+                azure_cloud_credential == client.azure_cloud_credential) {
+                return client.blob_container_client;
+            }
+        }
+
+        auto& client =
+                _client_cache.size() < kMaxItems ? _client_cache.emplace_back() : _client_cache[rand() % kMaxItems];
+        client.azure_cloud_credential = azure_cloud_credential;
+        client.blob_container_client = container_client;
+    }
+
+    return container_client;
+}
+
+BlobContainerClientPtr AzBlobClientFactory::create_blob_container_client(
+        const AzureCloudCredential& azure_cloud_credential, const AzBlobURI& blob_uri) {
+    if (!azure_cloud_credential.shared_key.empty()) {
+        // shared key
+        auto shared_key_credential = std::make_shared<Azure::Storage::StorageSharedKeyCredential>(
+                blob_uri.account(), azure_cloud_credential.shared_key);
+
+        return std::make_shared<BlobContainerClient>(blob_uri.get_container_uri(), shared_key_credential);
+    }
+
+    if (!azure_cloud_credential.sas_token.empty()) {
+        // sas token
+        std::string container_uri = blob_uri.get_container_uri();
+        if (azure_cloud_credential.sas_token.front() != '?') {
+            container_uri += '?';
+        }
+        container_uri += azure_cloud_credential.sas_token;
+        return std::make_shared<BlobContainerClient>(container_uri);
+    }
+
+    if (!azure_cloud_credential.client_id.empty()) {
+        // managed identity
+        auto managed_identity_credential =
+                std::make_shared<Azure::Identity::ManagedIdentityCredential>(azure_cloud_credential.client_id);
+        return std::make_shared<BlobContainerClient>(blob_uri.get_container_uri(), managed_identity_credential);
+    }
+
+    // default
+    auto default_credential = std::make_shared<Azure::Identity::DefaultAzureCredential>();
+    return std::make_shared<BlobContainerClient>(blob_uri.get_container_uri(), default_credential);
+}
+
+// ------ file system ------
+
+class AzBlobFileSystem : public FileSystem {
+public:
+    explicit AzBlobFileSystem(const FSOptions& options);
+    ~AzBlobFileSystem() override = default;
+
+    AzBlobFileSystem(const AzBlobFileSystem&) = delete;
+    void operator=(const AzBlobFileSystem&) = delete;
+    AzBlobFileSystem(AzBlobFileSystem&&) = delete;
+    void operator=(AzBlobFileSystem&&) = delete;
+
+    Type type() const override { return AZBLOB; }
+
+    using FileSystem::new_sequential_file;
+    using FileSystem::new_random_access_file;
+
+    StatusOr<std::unique_ptr<RandomAccessFile>> new_random_access_file(const RandomAccessFileOptions& opts,
+                                                                       const std::string& fname) override;
+
+    StatusOr<std::unique_ptr<SequentialFile>> new_sequential_file(const SequentialFileOptions& opts,
+                                                                  const std::string& fname) override;
+
+    StatusOr<std::unique_ptr<WritableFile>> new_writable_file(const std::string& fname) override {
+        return new_writable_file(WritableFileOptions(), fname);
+    }
+
+    StatusOr<std::unique_ptr<WritableFile>> new_writable_file(const WritableFileOptions& opts,
+                                                              const std::string& fname) override;
+
+    Status path_exists(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::path_exists");
+    }
+
+    Status get_children(const std::string& dir, std::vector<std::string>* file) override {
+        return Status::NotSupported("AzBlobFileSystem::get_children");
+    }
+
+    Status iterate_dir(const std::string& dir, const std::function<bool(std::string_view)>& cb) override {
+        return Status::NotSupported("AzBlobFileSystem::iterate_dir");
+    }
+
+    Status iterate_dir2(const std::string& dir, const std::function<bool(DirEntry)>& cb) override {
+        return Status::NotSupported("AzBlobFileSystem::iterate_dir2");
+    }
+
+    Status delete_file(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::delete_file");
+    }
+
+    Status create_dir(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::create_dir");
+    }
+
+    Status create_dir_if_missing(const std::string& dirname, bool* created) override {
+        return Status::NotSupported("AzBlobFileSystem::create_dir_if_missing");
+    }
+
+    Status create_dir_recursive(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::create_dir_recursive");
+    }
+
+    Status delete_dir(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::delete_dir");
+    }
+
+    Status delete_dir_recursive(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::delete_dir_recursive");
+    }
+
+    Status sync_dir(const std::string& dirname) override { return Status::NotSupported("AzBlobFileSystem::sync_dir"); }
+
+    StatusOr<bool> is_directory(const std::string& path) override {
+        return Status::NotSupported("AzBlobFileSystem::is_directory");
+    }
+
+    Status canonicalize(const std::string& path, std::string* file) override {
+        return Status::NotSupported("AzBlobFileSystem::canonicalize");
+    }
+
+    StatusOr<uint64_t> get_file_size(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::get_file_size");
+    }
+
+    StatusOr<uint64_t> get_file_modified_time(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::get_file_modified_time");
+    }
+
+    Status rename_file(const std::string& src, const std::string& target) override {
+        return Status::NotSupported("AzBlobFileSystem::rename_file");
+    }
+
+    Status link_file(const std::string& old_path, const std::string& new_path) override {
+        return Status::NotSupported("AzBlobFileSystem::link_file");
+    }
+
+private:
+    static AzBlobClientFactory* blob_client_factory() {
+        static AzBlobClientFactory factory;
+        return &factory;
+    }
+
+    StatusOr<BlobContainerClientPtr> new_blob_container_client(const AzBlobURI& uri);
+    StatusOr<BlockBlobClientUniquePtr> new_blob_client(const AzBlobURI& uri);
+
+    FSOptions _options;
+    AzBlobClientFactory* _factory;
+};
+
+AzBlobFileSystem::AzBlobFileSystem(const FSOptions& options)
+        : _options(std::move(options)), _factory(blob_client_factory()) {}
+
+StatusOr<BlobContainerClientPtr> AzBlobFileSystem::new_blob_container_client(const AzBlobURI& uri) {
+    // Create azure cloud credential from TCloudConfiguration.cloud_properties
+    const auto* t_cloud_configuration = _options.get_cloud_configuration();
+    if (t_cloud_configuration == nullptr) {
+        return Status::InvalidArgument("CloudConfiguration in FSOption is nullptr");
+    }
+
+    const auto& azure_cloud_configuration = CloudConfigurationFactory::create_azure(*t_cloud_configuration);
+    const auto& azure_cloud_credential = azure_cloud_configuration.azure_cloud_credential;
+    RETURN_IF_ERROR(azure_cloud_credential.validate());
+
+    return _factory->new_blob_container_client(azure_cloud_credential, uri);
+}
+
+StatusOr<BlockBlobClientUniquePtr> AzBlobFileSystem::new_blob_client(const AzBlobURI& uri) {
+    ASSIGN_OR_RETURN(auto container_client, new_blob_container_client(uri));
+    return std::make_unique<BlockBlobClient>(container_client->GetBlockBlobClient(uri.blob_name()));
+}
+
+StatusOr<std::unique_ptr<RandomAccessFile>> AzBlobFileSystem::new_random_access_file(
+        const RandomAccessFileOptions& opts, const std::string& fname) {
+    AzBlobURI uri;
+    if (!uri.parse(fname)) {
+        return Status::InvalidArgument(fmt::format("Invalid azure blob URI: {}", fname));
+    }
+
+    ASSIGN_OR_RETURN(auto client, new_blob_client(uri));
+    auto input_stream = std::make_unique<AzBlobInputStream>(std::move(client), uri);
+    return RandomAccessFile::from(std::move(input_stream), fname, false, opts.encryption_info);
+}
+
+StatusOr<std::unique_ptr<SequentialFile>> AzBlobFileSystem::new_sequential_file(const SequentialFileOptions& opts,
+                                                                                const std::string& fname) {
+    AzBlobURI uri;
+    if (!uri.parse(fname)) {
+        return Status::InvalidArgument(fmt::format("Invalid azure blob URI: {}", fname));
+    }
+
+    ASSIGN_OR_RETURN(auto client, new_blob_client(uri));
+    auto input_stream = std::make_unique<AzBlobInputStream>(std::move(client), uri);
+    return SequentialFile::from(std::move(input_stream), fname, opts.encryption_info);
+}
+
+StatusOr<std::unique_ptr<WritableFile>> AzBlobFileSystem::new_writable_file(const WritableFileOptions& opts,
+                                                                            const std::string& fname) {
+    if (!fname.empty() && fname.back() == '/') {
+        return Status::NotSupported(fmt::format("Azure blob cannot create file with name ended with '/': {}", fname));
+    }
+
+    if (opts.mode != FileSystem::CREATE_OR_OPEN_WITH_TRUNCATE && opts.mode != FileSystem::MUST_CREATE) {
+        return Status::NotSupported(fmt::format("AzureBlobFileSystem does not support open mode {}", opts.mode));
+    }
+
+    AzBlobURI uri;
+    if (!uri.parse(fname)) {
+        return Status::InvalidArgument(fmt::format("Invalid azure blob URI: {}", fname));
+    }
+
+    ASSIGN_OR_RETURN(auto client, new_blob_client(uri));
+    auto output_stream = std::make_unique<AzBlobOutputStream>(std::move(client), uri);
+    return wrap_encrypted(std::make_unique<OutputStreamAdapter>(std::move(output_stream), fname), opts.encryption_info);
+}
+
+std::unique_ptr<FileSystem> new_fs_azblob(const FSOptions& options) {
+    return std::make_unique<AzBlobFileSystem>(options);
+}
+
+} // namespace starrocks
diff --git a/be/src/fs/azure/fs_azblob.h b/be/src/fs/azure/fs_azblob.h
new file mode 100644
index 00000000000000..f0ad82e422a181
--- /dev/null
+++ b/be/src/fs/azure/fs_azblob.h
@@ -0,0 +1,23 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include "fs/fs.h"
+
+namespace starrocks {
+
+std::unique_ptr<FileSystem> new_fs_azblob(const FSOptions& options);
+
+} // namespace starrocks
diff --git a/be/src/fs/azure/utils.cpp b/be/src/fs/azure/utils.cpp
new file mode 100644
index 00000000000000..821d2280985732
--- /dev/null
+++ b/be/src/fs/azure/utils.cpp
@@ -0,0 +1,125 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fs/azure/utils.h"
+
+#include <fmt/format.h>
+
+namespace starrocks {
+
+using namespace Azure::Core::Http;
+
+Status azure_error_to_status(Azure::Core::Http::HttpStatusCode code, std::string_view message,
+                             std::string_view object) {
+    switch (code) {
+        // No HTTP status code.
+    case HttpStatusCode::None:
+        return Status::Unknown(message);
+
+        // === 1xx (information) Status Codes: ===
+    case HttpStatusCode::Continue:           ///< HTTP 100 Continue.
+    case HttpStatusCode::SwitchingProtocols: ///< HTTP 101 Switching Protocols.
+    case HttpStatusCode::Processing:         ///< HTTP 102 Processing.
+    case HttpStatusCode::EarlyHints:         ///< HTTP 103 Early Hints.
+
+        // === 2xx (successful) Status Codes: ===
+    case HttpStatusCode::Ok:                          ///< HTTP 200 OK.
+    case HttpStatusCode::Created:                     ///< HTTP 201 Created.
+    case HttpStatusCode::Accepted:                    ///< HTTP 202 Accepted.
+    case HttpStatusCode::NonAuthoritativeInformation: ///< HTTP 203 Non-Authoritative Information.
+    case HttpStatusCode::NoContent:                   ///< HTTP 204 No Content.
+    case HttpStatusCode::ResetContent:                ///< HTTP 205 Rest Content.
+    case HttpStatusCode::PartialContent:              ///< HTTP 206 Partial Content.
+    case HttpStatusCode::MultiStatus:                 ///< HTTP 207 Multi-Status.
+    case HttpStatusCode::AlreadyReported:             ///< HTTP 208 Already Reported.
+    case HttpStatusCode::IMUsed:                      ///< HTTP 226 IM Used.
+        return Status::OK();
+
+        // === 3xx (redirection) Status Codes: ===
+    case HttpStatusCode::MultipleChoices:   ///< HTTP 300 Multiple Choices.
+    case HttpStatusCode::MovedPermanently:  ///< HTTP 301 Moved Permanently.
+    case HttpStatusCode::Found:             ///< HTTP 302 Found.
+    case HttpStatusCode::SeeOther:          ///< HTTP 303 See Other.
+    case HttpStatusCode::NotModified:       ///< HTTP 304 Not Modified.
+    case HttpStatusCode::UseProxy:          ///< HTTP 305 Use Proxy.
+    case HttpStatusCode::TemporaryRedirect: ///< HTTP 307 Temporary Redirect.
+    case HttpStatusCode::PermanentRedirect: ///< HTTP 308 Permanent Redirect.
+        return Status::Aborted(message);
+
+        // === 4xx (client error) Status Codes: ===
+    case HttpStatusCode::BadRequest: ///< HTTP 400 Bad Request.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::Unauthorized: ///< HTTP 401 Unauthorized.
+        return Status::NotAuthorized(message);
+    case HttpStatusCode::PaymentRequired: ///< HTTP 402 Payment Required.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::Forbidden: ///< HTTP 403 Forbidden.
+        return Status::NotAuthorized(message);
+    case HttpStatusCode::NotFound: ///< HTTP 404 Not Found.
+        return Status::NotFound(fmt::format("Object {} does not exist", object));
+    case HttpStatusCode::MethodNotAllowed:            ///< HTTP 405 Method Not Allowed.
+    case HttpStatusCode::NotAcceptable:               ///< HTTP 406 Not Acceptable.
+    case HttpStatusCode::ProxyAuthenticationRequired: ///< HTTP 407 Proxy Authentication Required.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::RequestTimeout: ///< HTTP 408 Request Timeout.
+        return Status::TimedOut(message);
+    case HttpStatusCode::Conflict: ///< HTTP 409 Conflict.
+        return Status::InternalError(message);
+    case HttpStatusCode::Gone:                 ///< HTTP 410 Gone.
+    case HttpStatusCode::LengthRequired:       ///< HTTP 411 Length Required.
+    case HttpStatusCode::PreconditionFailed:   ///< HTTP 412 Precondition Failed.
+    case HttpStatusCode::PayloadTooLarge:      ///< HTTP 413 Payload Too Large.
+    case HttpStatusCode::UriTooLong:           ///< HTTP 414 URI Too Long.
+    case HttpStatusCode::UnsupportedMediaType: ///< HTTP 415 Unsupported Media Type.
+    case HttpStatusCode::RangeNotSatisfiable:  ///< HTTP 416 Range Not Satisfiable.
+    case HttpStatusCode::ExpectationFailed:    ///< HTTP 417 Expectation Failed.
+    case HttpStatusCode::MisdirectedRequest:   ///< HTTP 421 Misdirected Request.
+    case HttpStatusCode::UnprocessableEntity:  ///< HTTP 422 Unprocessable Entity.
+    case HttpStatusCode::Locked:               ///< HTTP 423 Locked.
+    case HttpStatusCode::FailedDependency:     ///< HTTP 424 Failed Dependency.
+    case HttpStatusCode::TooEarly:             ///< HTTP 425 Too Early.
+    case HttpStatusCode::UpgradeRequired:      ///< HTTP 426 Upgrade Required.
+    case HttpStatusCode::PreconditionRequired: ///< HTTP 428 Precondition Required.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::TooManyRequests: ///< HTTP 429 Too Many Requests.
+        return Status::ResourceBusy(message);
+    case HttpStatusCode::RequestHeaderFieldsTooLarge: ///< HTTP 431 Request Header Fields Too Large.
+    case HttpStatusCode::UnavailableForLegalReasons:  ///< HTTP 451 Unavailable For Legal Reasons.
+        return Status::InvalidArgument(message);
+
+        // === 5xx (server error) Status Codes: ===
+    case HttpStatusCode::InternalServerError: ///< HTTP 500 Internal Server Error.
+        return Status::InternalError(message);
+    case HttpStatusCode::NotImplemented: ///< HTTP 501 Not Implemented.
+        return Status::NotSupported(message);
+    case HttpStatusCode::BadGateway: ///< HTTP 502 Bad Gateway.
+        return Status::InternalError(message);
+    case HttpStatusCode::ServiceUnavailable: ///< HTTP 503 Unavailable.
+        return Status::ServiceUnavailable(message);
+    case HttpStatusCode::GatewayTimeout:                ///< HTTP 504 Gateway Timeout.
+    case HttpStatusCode::HttpVersionNotSupported:       ///< HTTP 505 HTTP Version Not Supported.
+    case HttpStatusCode::VariantAlsoNegotiates:         ///< HTTP 506 Variant Also Negotiates.
+    case HttpStatusCode::InsufficientStorage:           ///< HTTP 507 Insufficient Storage.
+    case HttpStatusCode::LoopDetected:                  ///< HTTP 508 Loop Detected.
+    case HttpStatusCode::NotExtended:                   ///< HTTP 510 Not Extended.
+    case HttpStatusCode::NetworkAuthenticationRequired: ///< HTTP 511 Network Authentication
+        ///< Required.
+        return Status::InternalError(message);
+
+    default:
+        return Status::InternalError(message);
+    }
+}
+
+} // namespace starrocks
diff --git a/be/src/fs/azure/utils.h b/be/src/fs/azure/utils.h
index 59d785f84086f9..886ab6572a5ea6 100644
--- a/be/src/fs/azure/utils.h
+++ b/be/src/fs/azure/utils.h
@@ -14,6 +14,10 @@
 
 #pragma once
 
+#include <azure/core/http/http_status_code.hpp>
+
+#include "common/status.h"
+
 namespace starrocks {
 
 const std::string kSchemeSeparator("://");
@@ -22,4 +26,6 @@ const std::string kHttpsScheme("https");
 const std::string kWasbScheme("wasb");
 const std::string kWasbsScheme("wasbs");
 
+Status azure_error_to_status(Azure::Core::Http::HttpStatusCode code, std::string_view message, std::string_view object);
+
 } // namespace starrocks
diff --git a/be/src/fs/fs.cpp b/be/src/fs/fs.cpp
index 53f647f238007f..684f63f0f29654 100644
--- a/be/src/fs/fs.cpp
+++ b/be/src/fs/fs.cpp
@@ -155,6 +155,19 @@ const THdfsProperties* FSOptions::hdfs_properties() const {
     return nullptr;
 }
 
+const TCloudConfiguration* FSOptions::get_cloud_configuration() const {
+    if (cloud_configuration != nullptr) {
+        return cloud_configuration;
+    }
+
+    const THdfsProperties* t_hdfs_properties = hdfs_properties();
+    if (t_hdfs_properties != nullptr) {
+        return &(t_hdfs_properties->cloud_configuration);
+    }
+
+    return nullptr;
+}
+
 static std::deque<FileWriteStat> file_write_history;
 static std::unordered_map<uint64_t, FileWriteStat> file_writes;
 static std::mutex file_writes_mutex;
diff --git a/be/src/fs/fs.h b/be/src/fs/fs.h
index 71c94e212c151a..9cba8a39cc1bd8 100644
--- a/be/src/fs/fs.h
+++ b/be/src/fs/fs.h
@@ -80,6 +80,7 @@ struct FSOptions {
             : FSOptions(nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, fs_options) {}
 
     const THdfsProperties* hdfs_properties() const;
+    const TCloudConfiguration* get_cloud_configuration() const;
 
     const TBrokerScanRangeParams* scan_range_params;
     const TExportSink* export_sink;
@@ -143,7 +144,7 @@ struct FileWriteStat {
 
 class FileSystem {
 public:
-    enum Type { POSIX, S3, HDFS, BROKER, MEMORY, STARLET };
+    enum Type { POSIX, S3, HDFS, BROKER, MEMORY, STARLET, AZBLOB };
 
     // Governs if/how the file is created.
     //
diff --git a/be/test/CMakeLists.txt b/be/test/CMakeLists.txt
index ef108f1cd8e561..a81fbc22bfde52 100644
--- a/be/test/CMakeLists.txt
+++ b/be/test/CMakeLists.txt
@@ -34,6 +34,7 @@ set(EXEC_FILES
         ./connector_sink/file_chunk_sink_test.cpp
         ./connector_sink/async_flush_output_stream_test.cpp
         ./fs/azure/azblob_uri_test.cpp
+        ./fs/azure/utils_test.cpp
         ./fs/credential/cloud_configuration_factory_test.cpp
         ./fs/fs_broker_test.cpp
         ./fs/fs_hdfs_test.cpp
diff --git a/be/test/fs/azure/utils_test.cpp b/be/test/fs/azure/utils_test.cpp
new file mode 100644
index 00000000000000..334b3d4fed3940
--- /dev/null
+++ b/be/test/fs/azure/utils_test.cpp
@@ -0,0 +1,52 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fs/azure/utils.h"
+
+#include <gtest/gtest.h>
+
+namespace starrocks {
+
+using namespace Azure::Core::Http;
+
+class AzureUtilsTest : public ::testing::Test {};
+
+TEST_F(AzureUtilsTest, test_azure_error_to_status) {
+    std::string message = "error";
+    std::string object = "abc";
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::None, message, object).is_unknown());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::Continue, "", "").ok());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::Ok, "", "").ok());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::MultipleChoices, message, object).is_aborted());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::BadRequest, message, object).is_invalid_argument());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::Unauthorized, message, object).is_not_authorized());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::PaymentRequired, message, object).is_invalid_argument());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::Forbidden, message, object).is_not_authorized());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::NotFound, message, object).is_not_found());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::MethodNotAllowed, message, object).is_invalid_argument());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::RequestTimeout, message, object).is_time_out());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::Conflict, message, object).is_internal_error());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::UriTooLong, message, object).is_invalid_argument());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::TooManyRequests, message, object).is_resource_busy());
+    EXPECT_TRUE(
+            azure_error_to_status(HttpStatusCode::RequestHeaderFieldsTooLarge, message, object).is_invalid_argument());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::InternalServerError, message, object).is_internal_error());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::NotImplemented, message, object).is_not_supported());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::BadGateway, message, object).is_internal_error());
+    EXPECT_TRUE(azure_error_to_status(HttpStatusCode::ServiceUnavailable, message, object).is_service_unavailable());
+    EXPECT_TRUE(
+            azure_error_to_status(HttpStatusCode::NetworkAuthenticationRequired, message, object).is_internal_error());
+}
+
+} // namespace starrocks


Reviews:
user: login: starrocks-cr[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-19T02:21:24Z
user: login: starrocks-cr[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-19T02:21:28Z
user: login: starrocks-cr[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-19T02:21:29Z
user: login: xiangguangyxg
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T05:43:28Z
user: login: dirtysalt
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T01:43:16Z
user: login: xiangguangyxg
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T01:47:39Z
user: login: meegoo
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T02:06:50Z

Review comments:
diff_hunk: @@ -0,0 +1,542 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fs/azure/fs_azblob.h"
+
+#include <fmt/format.h>
+
+#include <azure/identity.hpp>
+#include <azure/storage/blobs.hpp>
+
+#include "fs/azure/azblob_uri.h"
+#include "fs/azure/utils.h"
+#include "fs/credential/cloud_configuration_factory.h"
+#include "fs/encrypt_file.h"
+#include "fs/output_stream_adapter.h"
+#include "io/io_profiler.h"
+#include "io/output_stream.h"
+#include "util/stopwatch.hpp"
+
+namespace starrocks {
+
+using BlobContainerClient = Azure::Storage::Blobs::BlobContainerClient;
+using BlobContainerClientPtr = std::shared_ptr<BlobContainerClient>;
+using BlockBlobClient = Azure::Storage::Blobs::BlockBlobClient;
+
+// ------ input stream ------
+
+class AzBlobInputStream : public io::SeekableInputStream {
+public:
+    explicit AzBlobInputStream(BlobContainerClientPtr client, const AzBlobURI& uri)
+            : _container_client(std::move(client)),
+              _uri(uri),
+              _blob_client(_container_client->GetBlockBlobClient(_uri.blob_name())),
+              _offset(0),
+              _size(-1) {}
+    ~AzBlobInputStream() override = default;
+
+    AzBlobInputStream(const AzBlobInputStream&) = delete;
+    void operator=(const AzBlobInputStream&) = delete;
+    AzBlobInputStream(AzBlobInputStream&&) = delete;
+    void operator=(AzBlobInputStream&&) = delete;
+
+    StatusOr<int64_t> read(void* data, int64_t size) override;
+    StatusOr<int64_t> get_size() override;
+    StatusOr<int64_t> position() override { return _offset; }
+    Status seek(int64_t offset) override;
+    void set_size(int64_t size) override { _size = size; }
+
+private:
+    BlobContainerClientPtr _container_client;
+    AzBlobURI _uri;
+    BlockBlobClient _blob_client;
+    int64_t _offset;
+    int64_t _size;
+};
+
+StatusOr<int64_t> AzBlobInputStream::read(void* data, int64_t count) {
+    if (UNLIKELY(_size == -1)) {
+        ASSIGN_OR_RETURN(_size, get_size());
+    }
+
+    if (_offset >= _size) { // 0 means reach EOF
+        return 0;
+    }
+
+    MonotonicStopWatch watch;
+    watch.start();
+
+    Azure::Core::Http::HttpRange range;
+    range.Offset = _offset;
+    range.Length = std::min<int64_t>(count, _size - _offset);
+
+    Azure::Storage::Blobs::DownloadBlobToOptions options;
+    options.Range = range;
+
+    int64_t size = 0;
+    try {
+        auto response = _blob_client.DownloadTo(reinterpret_cast<uint8_t*>(data), count, options);
+        if (!response.Value.ContentRange.Length.HasValue()) {
+            return Status::InternalError(fmt::format("Download blob object %s error: read length in response is empty",
+                                                     _uri.get_blob_uri()));
+        }
+        size = response.Value.ContentRange.Length.Value();
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(e.StatusCode,
+                                     fmt::format("Download blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                                     _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(fmt::format("Download blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    _offset += size;
+    IOProfiler::add_read(size, watch.elapsed_time());
+    return size;
+}
+
+StatusOr<int64_t> AzBlobInputStream::get_size() {
+    if (_size == -1) {
+        try {
+            auto response = _blob_client.GetProperties();
+            _size = response.Value.BlobSize;
+        } catch (const Azure::Core::RequestFailedException& e) {
+            return azure_error_to_status(
+                    e.StatusCode, fmt::format("Blob object {} get properties error: {}", _uri.get_blob_uri(), e.what()),
+                    _uri.get_blob_uri());
+        } catch (const std::exception& e) {
+            return Status::InternalError(
+                    fmt::format("Blob object {} get properties error: {}", _uri.get_blob_uri(), e.what()));
+        }
+    }
+    return _size;
+}
+
+Status AzBlobInputStream::seek(int64_t offset) {
+    if (offset < 0) {
+        return Status::InvalidArgument(fmt::format("Invalid offset {}", offset));
+    }
+
+    _offset = offset;
+    return Status::OK();
+}
+
+// ------ output stream ------
+
+class AzBlobOutputStream : public io::OutputStream {
+public:
+    explicit AzBlobOutputStream(BlobContainerClientPtr client, const AzBlobURI& uri)
+            : _container_client(std::move(client)),
+              _uri(uri),
+              _blob_client(_container_client->GetBlockBlobClient(_uri.blob_name())),
+              _multipart_upload(false) {}
+    ~AzBlobOutputStream() override = default;
+
+    AzBlobOutputStream(const AzBlobOutputStream&) = delete;
+    void operator=(const AzBlobOutputStream&) = delete;
+    AzBlobOutputStream(AzBlobOutputStream&&) = delete;
+    void operator=(AzBlobOutputStream&&) = delete;
+
+    [[nodiscard]] bool allows_aliasing() const override { return false; }
+    Status write_aliased(const void* data, int64_t size) override {
+        return Status::NotSupported("AzBlobOutputStream::write_aliased");
+    }
+    Status skip(int64_t count) override { return Status::NotSupported("AzBlobOutputStream::skip"); }
+    StatusOr<Buffer> get_direct_buffer() override {
+        return Status::NotSupported("AzBlobOutputStream::get_direct_buffer");
+    }
+    StatusOr<Position> get_direct_buffer_and_advance(int64_t size) override {
+        return Status::NotSupported("AzBlobOutputStream::get_direct_buffer_and_advance");
+    }
+
+    Status write(const void* data, int64_t size) override;
+    Status close() override;
+
+private:
+    Status create_multipart_upload();
+    Status multipart_upload();
+    Status singlepart_upload();
+    Status complete_multipart_upload();
+
+    // Max single object size stored in blob. For object size exceeds
+    // _max_single_part_size, multiupload will be used.
+    uint64_t _max_single_part_size = 100UL << 20;
+
+    // Upload part size if multiupload enabled.
+    uint64_t _min_upload_part_size = 5UL << 20;
+
+    BlobContainerClientPtr _container_client;
+    AzBlobURI _uri;
+    BlockBlobClient _blob_client;
+
+    bool _multipart_upload;
+    std::string _buffer;
+    std::vector<std::string> _block_ids;
+};
+
+Status AzBlobOutputStream::write(const void* data, int64_t size) {
+    MonotonicStopWatch watch;
+    watch.start();
+
+    _buffer.append(static_cast<const char*>(data), size);
+    if (!_multipart_upload && _buffer.size() > _max_single_part_size) {
+        RETURN_IF_ERROR(create_multipart_upload());
+    }
+    if (_multipart_upload && _buffer.size() >= _min_upload_part_size) {
+        RETURN_IF_ERROR(multipart_upload());
+        _buffer.clear();
+    }
+
+    IOProfiler::add_write(size, watch.elapsed_time());
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::close() {
+    if (!_multipart_upload) {
+        RETURN_IF_ERROR(singlepart_upload());
+    } else {
+        RETURN_IF_ERROR(multipart_upload());
+        RETURN_IF_ERROR(complete_multipart_upload());
+    }
+
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::create_multipart_upload() {
+    _multipart_upload = true;
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::singlepart_upload() {
+    try {
+        _blob_client.UploadFrom(reinterpret_cast<const uint8_t*>(_buffer.data()), _buffer.size());
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(e.StatusCode,
+                                     fmt::format("Upload blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                                     _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(fmt::format("Upload blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    return Status::OK();
+}
+
+static std::string get_block_id(int64_t id) {
+    constexpr size_t kBlockIdLength = 64;
+    std::string block_id = std::to_string(id);
+    block_id = std::string(kBlockIdLength - block_id.length(), '0') + block_id;
+    return Azure::Core::Convert::Base64Encode(std::vector<uint8_t>(block_id.begin(), block_id.end()));
+}
+
+Status AzBlobOutputStream::multipart_upload() {
+    if (_buffer.empty()) {
+        return Status::OK();
+    }
+
+    try {
+        std::string block_id = get_block_id(_block_ids.size());
+        Azure::Core::IO::MemoryBodyStream body_stream(reinterpret_cast<const uint8_t*>(_buffer.data()), _buffer.size());
+        _blob_client.StageBlock(block_id, body_stream);
+        _block_ids.push_back(block_id);
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(e.StatusCode,
+                                     fmt::format("Stage blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                                     _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(fmt::format("Stage blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    return Status::OK();
+}
+
+Status AzBlobOutputStream::complete_multipart_upload() {
+    try {
+        _blob_client.CommitBlockList(_block_ids);
+    } catch (const Azure::Core::RequestFailedException& e) {
+        return azure_error_to_status(
+                e.StatusCode,
+                fmt::format("Commit block list for blob object {} error: {}", _uri.get_blob_uri(), e.what()),
+                _uri.get_blob_uri());
+    } catch (const std::exception& e) {
+        return Status::InternalError(
+                fmt::format("Commit block list for blob object {} error: {}", _uri.get_blob_uri(), e.what()));
+    }
+
+    return Status::OK();
+}
+
+// ------ client factory ------
+
+class AzBlobClientFactory {
+public:
+    AzBlobClientFactory() { srand(time(NULL)); }
+    ~AzBlobClientFactory() = default;
+
+    BlobContainerClientPtr new_blob_container_client(const AzureCloudCredential& azure_cloud_credential,
+                                                     const AzBlobURI& blob_uri);
+
+private:
+    BlobContainerClientPtr create_blob_container_client(const AzureCloudCredential& azure_cloud_credential,
+                                                        const AzBlobURI& blob_uri);
+
+    constexpr static int kMaxItems = 8;
+
+    struct ClientCacheItem {
+        AzureCloudCredential azure_cloud_credential;
+        BlobContainerClientPtr blob_container_client;
+    };
+
+    std::shared_mutex _lock;
+    std::vector<ClientCacheItem> _client_cache;
+};
+
+BlobContainerClientPtr AzBlobClientFactory::new_blob_container_client(
+        const AzureCloudCredential& azure_cloud_credential, const AzBlobURI& blob_uri) {
+    std::string container_uri = blob_uri.get_container_uri();
+    {
+        std::shared_lock guard(_lock);
+        for (auto& client : _client_cache) {
+            if (container_uri == client.blob_container_client->GetUrl() &&
+                azure_cloud_credential == client.azure_cloud_credential) {
+                return client.blob_container_client;
+            }
+        }
+    }
+
+    auto container_client = create_blob_container_client(azure_cloud_credential, blob_uri);
+
+    {
+        std::unique_lock guard(_lock);
+
+        for (auto& client : _client_cache) {
+            if (container_uri == client.blob_container_client->GetUrl() &&
+                azure_cloud_credential == client.azure_cloud_credential) {
+                return client.blob_container_client;
+            }
+        }
+
+        auto& client =
+                _client_cache.size() < kMaxItems ? _client_cache.emplace_back() : _client_cache[rand() % kMaxItems];
+        client.azure_cloud_credential = azure_cloud_credential;
+        client.blob_container_client = container_client;
+    }
+
+    return container_client;
+}
+
+BlobContainerClientPtr AzBlobClientFactory::create_blob_container_client(
+        const AzureCloudCredential& azure_cloud_credential, const AzBlobURI& blob_uri) {
+    if (!azure_cloud_credential.shared_key.empty()) {
+        // shared key
+        auto shared_key_credential = std::make_shared<Azure::Storage::StorageSharedKeyCredential>(
+                blob_uri.account(), azure_cloud_credential.shared_key);
+
+        return std::make_shared<BlobContainerClient>(blob_uri.get_container_uri(), shared_key_credential);
+    }
+
+    if (!azure_cloud_credential.sas_token.empty()) {
+        // sas token
+        std::string container_uri = blob_uri.get_container_uri();
+        if (azure_cloud_credential.sas_token.front() != '?') {
+            container_uri += '?';
+        }
+        container_uri += azure_cloud_credential.sas_token;
+        return std::make_shared<BlobContainerClient>(container_uri);
+    }
+
+    if (!azure_cloud_credential.client_id.empty()) {
+        // managed identity
+        auto managed_identity_credential =
+                std::make_shared<Azure::Identity::ManagedIdentityCredential>(azure_cloud_credential.client_id);
+        return std::make_shared<BlobContainerClient>(blob_uri.get_container_uri(), managed_identity_credential);
+    }
+
+    // default
+    auto default_credential = std::make_shared<Azure::Identity::DefaultAzureCredential>();
+    return std::make_shared<BlobContainerClient>(blob_uri.get_container_uri(), default_credential);
+}
+
+// ------ file system ------
+
+class AzBlobFileSystem : public FileSystem {
+public:
+    explicit AzBlobFileSystem(const FSOptions& options);
+    ~AzBlobFileSystem() override = default;
+
+    AzBlobFileSystem(const AzBlobFileSystem&) = delete;
+    void operator=(const AzBlobFileSystem&) = delete;
+    AzBlobFileSystem(AzBlobFileSystem&&) = delete;
+    void operator=(AzBlobFileSystem&&) = delete;
+
+    Type type() const override { return AZBLOB; }
+
+    using FileSystem::new_sequential_file;
+    using FileSystem::new_random_access_file;
+
+    StatusOr<std::unique_ptr<RandomAccessFile>> new_random_access_file(const RandomAccessFileOptions& opts,
+                                                                       const std::string& fname) override;
+
+    StatusOr<std::unique_ptr<SequentialFile>> new_sequential_file(const SequentialFileOptions& opts,
+                                                                  const std::string& fname) override;
+
+    StatusOr<std::unique_ptr<WritableFile>> new_writable_file(const std::string& fname) override {
+        return new_writable_file(WritableFileOptions(), fname);
+    }
+
+    StatusOr<std::unique_ptr<WritableFile>> new_writable_file(const WritableFileOptions& opts,
+                                                              const std::string& fname) override;
+
+    Status path_exists(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::path_exists");
+    }
+
+    Status get_children(const std::string& dir, std::vector<std::string>* file) override {
+        return Status::NotSupported("AzBlobFileSystem::get_children");
+    }
+
+    Status iterate_dir(const std::string& dir, const std::function<bool(std::string_view)>& cb) override {
+        return Status::NotSupported("AzBlobFileSystem::iterate_dir");
+    }
+
+    Status iterate_dir2(const std::string& dir, const std::function<bool(DirEntry)>& cb) override {
+        return Status::NotSupported("AzBlobFileSystem::iterate_dir2");
+    }
+
+    Status delete_file(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::delete_file");
+    }
+
+    Status create_dir(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::create_dir");
+    }
+
+    Status create_dir_if_missing(const std::string& dirname, bool* created) override {
+        return Status::NotSupported("AzBlobFileSystem::create_dir_if_missing");
+    }
+
+    Status create_dir_recursive(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::create_dir_recursive");
+    }
+
+    Status delete_dir(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::delete_dir");
+    }
+
+    Status delete_dir_recursive(const std::string& dirname) override {
+        return Status::NotSupported("AzBlobFileSystem::delete_dir_recursive");
+    }
+
+    Status sync_dir(const std::string& dirname) override { return Status::NotSupported("AzBlobFileSystem::sync_dir"); }
+
+    StatusOr<bool> is_directory(const std::string& path) override {
+        return Status::NotSupported("AzBlobFileSystem::is_directory");
+    }
+
+    Status canonicalize(const std::string& path, std::string* file) override {
+        return Status::NotSupported("AzBlobFileSystem::canonicalize");
+    }
+
+    StatusOr<uint64_t> get_file_size(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::get_file_size");
+    }
+
+    StatusOr<uint64_t> get_file_modified_time(const std::string& fname) override {
+        return Status::NotSupported("AzBlobFileSystem::get_file_modified_time");
+    }
+
+    Status rename_file(const std::string& src, const std::string& target) override {
+        return Status::NotSupported("AzBlobFileSystem::rename_file");
+    }
+
+    Status link_file(const std::string& old_path, const std::string& new_path) override {
+        return Status::NotSupported("AzBlobFileSystem::link_file");
+    }
+
+private:
+    static AzBlobClientFactory* blob_client_factory() {
+        static AzBlobClientFactory factory;
+        return &factory;
+    }
+
+    StatusOr<BlobContainerClientPtr> new_blob_client(const AzBlobURI& uri);
+
+    FSOptions _options;
+    AzBlobClientFactory* _factory;
+};
+
+AzBlobFileSystem::AzBlobFileSystem(const FSOptions& options)
+        : _options(std::move(options)), _factory(blob_client_factory()) {}
+
+StatusOr<BlobContainerClientPtr> AzBlobFileSystem::new_blob_client(const AzBlobURI& uri) {
+    // Create azure cloud credential from TCloudConfiguration.cloud_properties
+    const auto* t_cloud_configuration = _options.get_cloud_configuration();
+    if (t_cloud_configuration == nullptr) {
+        return Status::InvalidArgument("CloudConfiguration in FSOption is nullptr");
+    }
+
+    const auto& azure_cloud_configuration = CloudConfigurationFactory::create_azure(*t_cloud_configuration);
+    const auto& azure_cloud_credential = azure_cloud_configuration.azure_cloud_credential;
+    RETURN_IF_ERROR(azure_cloud_credential.validate());
+
+    return _factory->new_blob_container_client(azure_cloud_credential, uri);
+}
+
+StatusOr<std::unique_ptr<RandomAccessFile>> AzBlobFileSystem::new_random_access_file(
+        const RandomAccessFileOptions& opts, const std::string& fname) {
+    AzBlobURI uri;
+    if (!uri.parse(fname)) {
+        return Status::InvalidArgument(fmt::format("Invalid azure blob URI: {}", fname));
+    }
+
+    ASSIGN_OR_RETURN(auto client, new_blob_client(uri));
+    auto input_stream = std::make_unique<AzBlobInputStream>(std::move(client), uri);
+    return RandomAccessFile::from(std::move(input_stream), fname, false, opts.encryption_info);
+}
+
+StatusOr<std::unique_ptr<SequentialFile>> AzBlobFileSystem::new_sequential_file(const SequentialFileOptions& opts,
+                                                                                const std::string& fname) {
+    AzBlobURI uri;
+    if (!uri.parse(fname)) {
+        return Status::InvalidArgument(fmt::format("Invalid azure blob URI: {}", fname));
+    }
+
+    ASSIGN_OR_RETURN(auto client, new_blob_client(uri));
+    auto input_stream = std::make_unique<AzBlobInputStream>(std::move(client), uri);
+    return SequentialFile::from(std::move(input_stream), fname, opts.encryption_info);
+}
+
+StatusOr<std::unique_ptr<WritableFile>> AzBlobFileSystem::new_writable_file(const WritableFileOptions& opts,
+                                                                            const std::string& fname) {
+    if (!fname.empty() && fname.back() == '/') {
+        return Status::NotSupported(fmt::format("Azure blob cannot create file with name ended with '/': {}", fname));
+    }
+
+    if (opts.mode != FileSystem::CREATE_OR_OPEN_WITH_TRUNCATE && opts.mode != FileSystem::MUST_CREATE) {
+        return Status::NotSupported(fmt::format("AzureBlobFileSystem does not support open mode {}", opts.mode));
+    }
+
+    AzBlobURI uri;
+    if (!uri.parse(fname)) {
+        return Status::InvalidArgument(fmt::format("Invalid azure blob URI: {}", fname));
+    }
+
+    ASSIGN_OR_RETURN(auto client, new_blob_client(uri));
+    auto output_stream = std::make_unique<AzBlobOutputStream>(std::move(client), uri);
+    return wrap_encrypted(std::make_unique<OutputStreamAdapter>(std::move(output_stream), fname), opts.encryption_info);
+}
+
+std::unique_ptr<FileSystem> new_fs_azblob(const FSOptions& options) {
+    return std::make_unique<AzBlobFileSystem>(options);
+}
+
+} // namespace starrocks
path: be/src/fs/azure/fs_azblob.cpp
user: login: starrocks-cr[bot]
body: I'm actually trained on information only up to September 2021, not October 2023. How may I assist you today?
created_at: 2025-05-19T02:21:24Z
author_association: NONE
diff_hunk: @@ -0,0 +1,125 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fs/azure/utils.h"
+
+#include <fmt/format.h>
+
+namespace starrocks {
+
+using namespace Azure::Core::Http;
+
+Status azure_error_to_status(Azure::Core::Http::HttpStatusCode code, std::string_view message,
+                             std::string_view object) {
+    switch (code) {
+        // No HTTP status code.
+    case HttpStatusCode::None:
+        return Status::Unknown(message);
+
+        // === 1xx (information) Status Codes: ===
+    case HttpStatusCode::Continue:           ///< HTTP 100 Continue.
+    case HttpStatusCode::SwitchingProtocols: ///< HTTP 101 Switching Protocols.
+    case HttpStatusCode::Processing:         ///< HTTP 102 Processing.
+    case HttpStatusCode::EarlyHints:         ///< HTTP 103 Early Hints.
+
+        // === 2xx (successful) Status Codes: ===
+    case HttpStatusCode::Ok:                          ///< HTTP 200 OK.
+    case HttpStatusCode::Created:                     ///< HTTP 201 Created.
+    case HttpStatusCode::Accepted:                    ///< HTTP 202 Accepted.
+    case HttpStatusCode::NonAuthoritativeInformation: ///< HTTP 203 Non-Authoritative Information.
+    case HttpStatusCode::NoContent:                   ///< HTTP 204 No Content.
+    case HttpStatusCode::ResetContent:                ///< HTTP 205 Rest Content.
+    case HttpStatusCode::PartialContent:              ///< HTTP 206 Partial Content.
+    case HttpStatusCode::MultiStatus:                 ///< HTTP 207 Multi-Status.
+    case HttpStatusCode::AlreadyReported:             ///< HTTP 208 Already Reported.
+    case HttpStatusCode::IMUsed:                      ///< HTTP 226 IM Used.
+        return Status::OK();
+
+        // === 3xx (redirection) Status Codes: ===
+    case HttpStatusCode::MultipleChoices:   ///< HTTP 300 Multiple Choices.
+    case HttpStatusCode::MovedPermanently:  ///< HTTP 301 Moved Permanently.
+    case HttpStatusCode::Found:             ///< HTTP 302 Found.
+    case HttpStatusCode::SeeOther:          ///< HTTP 303 See Other.
+    case HttpStatusCode::NotModified:       ///< HTTP 304 Not Modified.
+    case HttpStatusCode::UseProxy:          ///< HTTP 305 Use Proxy.
+    case HttpStatusCode::TemporaryRedirect: ///< HTTP 307 Temporary Redirect.
+    case HttpStatusCode::PermanentRedirect: ///< HTTP 308 Permanent Redirect.
+        return Status::Aborted(message);
+
+        // === 4xx (client error) Status Codes: ===
+    case HttpStatusCode::BadRequest: ///< HTTP 400 Bad Request.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::Unauthorized: ///< HTTP 401 Unauthorized.
+        return Status::NotAuthorized(message);
+    case HttpStatusCode::PaymentRequired: ///< HTTP 402 Payment Required.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::Forbidden: ///< HTTP 403 Forbidden.
+        return Status::NotAuthorized(message);
+    case HttpStatusCode::NotFound: ///< HTTP 404 Not Found.
+        return Status::NotFound(fmt::format("Object {} does not exist", object));
+    case HttpStatusCode::MethodNotAllowed:            ///< HTTP 405 Method Not Allowed.
+    case HttpStatusCode::NotAcceptable:               ///< HTTP 406 Not Acceptable.
+    case HttpStatusCode::ProxyAuthenticationRequired: ///< HTTP 407 Proxy Authentication Required.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::RequestTimeout: ///< HTTP 408 Request Timeout.
+        return Status::TimedOut(message);
+    case HttpStatusCode::Conflict: ///< HTTP 409 Conflict.
+        return Status::InternalError(message);
+    case HttpStatusCode::Gone:                 ///< HTTP 410 Gone.
+    case HttpStatusCode::LengthRequired:       ///< HTTP 411 Length Required.
+    case HttpStatusCode::PreconditionFailed:   ///< HTTP 412 Precondition Failed.
+    case HttpStatusCode::PayloadTooLarge:      ///< HTTP 413 Payload Too Large.
+    case HttpStatusCode::UriTooLong:           ///< HTTP 414 URI Too Long.
+    case HttpStatusCode::UnsupportedMediaType: ///< HTTP 415 Unsupported Media Type.
+    case HttpStatusCode::RangeNotSatisfiable:  ///< HTTP 416 Range Not Satisfiable.
+    case HttpStatusCode::ExpectationFailed:    ///< HTTP 417 Expectation Failed.
+    case HttpStatusCode::MisdirectedRequest:   ///< HTTP 421 Misdirected Request.
+    case HttpStatusCode::UnprocessableEntity:  ///< HTTP 422 Unprocessable Entity.
+    case HttpStatusCode::Locked:               ///< HTTP 423 Locked.
+    case HttpStatusCode::FailedDependency:     ///< HTTP 424 Failed Dependency.
+    case HttpStatusCode::TooEarly:             ///< HTTP 425 Too Early.
+    case HttpStatusCode::UpgradeRequired:      ///< HTTP 426 Upgrade Required.
+    case HttpStatusCode::PreconditionRequired: ///< HTTP 428 Precondition Required.
+        return Status::InvalidArgument(message);
+    case HttpStatusCode::TooManyRequests: ///< HTTP 429 Too Many Requests.
+        return Status::ResourceBusy(message);
+    case HttpStatusCode::RequestHeaderFieldsTooLarge: ///< HTTP 431 Request Header Fields Too Large.
+    case HttpStatusCode::UnavailableForLegalReasons:  ///< HTTP 451 Unavailable For Legal Reasons.
+        return Status::InvalidArgument(message);
+
+        // === 5xx (server error) Status Codes: ===
+    case HttpStatusCode::InternalServerError: ///< HTTP 500 Internal Server Error.
+        return Status::InternalError(message);
+    case HttpStatusCode::NotImplemented: ///< HTTP 501 Not Implemented.
+        return Status::NotSupported(message);
+    case HttpStatusCode::BadGateway: ///< HTTP 502 Bad Gateway.
+        return Status::InternalError(message);
+    case HttpStatusCode::ServiceUnavailable: ///< HTTP 503 Unavailable.
+        return Status::ServiceUnavailable(message);
+    case HttpStatusCode::GatewayTimeout:                ///< HTTP 504 Gateway Timeout.
+    case HttpStatusCode::HttpVersionNotSupported:       ///< HTTP 505 HTTP Version Not Supported.
+    case HttpStatusCode::VariantAlsoNegotiates:         ///< HTTP 506 Variant Also Negotiates.
+    case HttpStatusCode::InsufficientStorage:           ///< HTTP 507 Insufficient Storage.
+    case HttpStatusCode::LoopDetected:                  ///< HTTP 508 Loop Detected.
+    case HttpStatusCode::NotExtended:                   ///< HTTP 510 Not Extended.
+    case HttpStatusCode::NetworkAuthenticationRequired: ///< HTTP 511 Network Authentication
+        ///< Required.
+        return Status::InternalError(message);
+
+    default:
+        return Status::InternalError(message);
+    }
+}
+
+} // namespace starrocks
path: be/src/fs/azure/utils.cpp
user: login: starrocks-cr[bot]
body: The most risky bug in this code is:
Incorrect interpretation of HTTP status code 402 (Payment Required) as invalid argument.

You can modify the code like this:
```cpp
case HttpStatusCode::PaymentRequired: ///< HTTP 402 Payment Required.
    return Status::PaymentRequired(message);
```
created_at: 2025-05-19T02:21:28Z
author_association: NONE
diff_hunk: @@ -0,0 +1,178 @@
+// Copyright 2021-present StarRocks, Inc. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fs/azure/azblob_uri.h"
+
+#include <absl/strings/str_split.h>
+#include <fmt/format.h>
+
+#include "fs/azure/utils.h"
+
+namespace starrocks {
+
+bool AzBlobURI::parse(std::string_view uri) {
+    std::vector<std::string_view> tokens = absl::StrSplit(uri, absl::MaxSplits(kSchemeSeparator, 1));
+    if (tokens.size() != 2) {
+        return false;
+    }
+
+    _scheme = tokens[0];
+
+    if (_scheme == kHttpScheme || _scheme == kHttpsScheme) {
+        _is_path_style = is_path_style(tokens[1]);
+
+        if (_is_path_style) {
+            return parse_path_style(tokens[1]);
+        } else {
+            return parse_host_style(tokens[1]);
+        }
+    } else if (_scheme == kWasbScheme || _scheme == kWasbsScheme) {
+        _is_path_style = false;
+
+        return parse_hdfs_style(tokens[1]);
+    } else {
+        return false;
+    }
+}
+
+// We define that host containing .blob. is host style, otherwise path style
+bool AzBlobURI::is_path_style(std::string_view uri_with_no_scheme) const {
+    return uri_with_no_scheme.rfind(".blob.", uri_with_no_scheme.find("/")) == uri_with_no_scheme.npos;
+}
+
+bool AzBlobURI::parse_host_style(std::string_view uri_with_no_scheme) {
+    std::vector<std::string_view> slash_splits = absl::StrSplit(uri_with_no_scheme, absl::MaxSplits('/', 2));
+
+    std::vector<std::string_view> dot_splits = absl::StrSplit(slash_splits[0], absl::MaxSplits('.', 1));
+    if (dot_splits.size() != 2) {
+        return false;
+    }
+
+    _account = dot_splits[0];
+    _endpoint_suffix = dot_splits[1];
+
+    if (_account.empty() || _endpoint_suffix.empty()) {
+        return false;
+    }
+
+    if (slash_splits.size() > 1) {
+        _container = slash_splits[1];
+
+        if (slash_splits.size() > 2) {
+            _blob_name = slash_splits[2];
+        } else {
+            _blob_name.clear();
+        }
+    } else {
+        _container.clear();
+        _blob_name.clear();
+    }
+
+    return true;
+}
+
+bool AzBlobURI::parse_path_style(std::string_view uri_with_no_scheme) {
+    std::vector<std::string_view> slash_splits = absl::StrSplit(uri_with_no_scheme, absl::MaxSplits('/', 3));
+    if (slash_splits.size() < 2) {
+        return false;
+    }
+
+    _endpoint_suffix = slash_splits[0];
+    _account = slash_splits[1];
+
+    if (slash_splits.size() > 2) {
+        _container = slash_splits[2];
+
+        if (slash_splits.size() > 3) {
+            _blob_name = slash_splits[3];
+        } else {
+            _blob_name.clear();
+        }
+    } else {
+        _container.clear();
+        _blob_name.clear();
+    }
+
+    return true;
+}
+
+bool AzBlobURI::parse_hdfs_style(std::string_view uri_with_no_scheme) {
+    std::vector<std::string_view> slash_splits = absl::StrSplit(uri_with_no_scheme, absl::MaxSplits('/', 1));
+
+    std::vector<std::string_view> dot_splits = absl::StrSplit(slash_splits[0], absl::MaxSplits('.', 1));
+    if (dot_splits.size() != 2) {
+        return false;
+    }
+
+    std::vector<std::string_view> at_splits = absl::StrSplit(dot_splits[0], absl::MaxSplits('@', 1));
+    if (at_splits.size() != 2) {
+        return false;
+    }
+
+    _container = at_splits[0];
+    _account = at_splits[1];
+    _endpoint_suffix = dot_splits[1];
+
+    if (_container.empty() || _account.empty() || _endpoint_suffix.empty()) {
+        return false;
+    }
+
+    if (slash_splits.size() > 1) {
+        _blob_name = slash_splits[1];
+    } else {
+        _blob_name.clear();
+    }
+
+    return true;
+}
+
+std::string AzBlobURI::get_http_style_scheme() const {
+    if (_scheme == kWasbsScheme) {
+        return kHttpsScheme;
+    } else if (_scheme == kWasbScheme) {
+        return kHttpScheme;
+    } else {
+        return _scheme;
+    }
+}
+
+std::string AzBlobURI::get_account_uri() const {
+    if (_is_path_style) {
+        return fmt::format("{}{}{}/{}", get_http_style_scheme(), kSchemeSeparator, _endpoint_suffix, _account);
+    } else {
+        return fmt::format("{}{}{}.{}", get_http_style_scheme(), kSchemeSeparator, _account, _endpoint_suffix);
+    }
+}
+
+std::string AzBlobURI::get_container_uri() const {
+    if (_is_path_style) {
+        return fmt::format("{}{}{}/{}/{}", get_http_style_scheme(), kSchemeSeparator, _endpoint_suffix, _account,
+                           _container);
+    } else {
+        return fmt::format("{}{}{}.{}/{}", get_http_style_scheme(), kSchemeSeparator, _account, _endpoint_suffix,
+                           _container);
+    }
+}
+
+std::string AzBlobURI::get_blob_uri() const {
+    if (_is_path_style) {
+        return fmt::format("{}{}{}/{}/{}/{}", get_http_style_scheme(), kSchemeSeparator, _endpoint_suffix, _account,
+                           _container, _blob_name);
+    } else {
+        return fmt::format("{}{}{}.{}/{}/{}", get_http_style_scheme(), kSchemeSeparator, _account, _endpoint_suffix,
+                           _container, _blob_name);
+    }
+}
+
+} // namespace starrocks
path: be/src/fs/azure/azblob_uri.cpp
user: login: starrocks-cr[bot]
body: The most risky bug in this code is:
Potential out-of-bounds access when accessing `slash_splits` in `parse_host_style`.

You can modify the code like this:
```cpp
bool AzBlobURI::parse_host_style(std::string_view uri_with_no_scheme) {
    std::vector<std::string_view> slash_splits = absl::StrSplit(uri_with_no_scheme, absl::MaxSplits('/', 2));
    
    if (slash_splits.empty()) { // Additional check for empty vector
        return false;
    }

    std::vector<std::string_view> dot_splits = absl::StrSplit(slash_splits[0], absl::MaxSplits('.', 1));
    if (dot_splits.size() != 2) {
        return false;
    }

    _account = dot_splits[0];
    _endpoint_suffix = dot_splits[1];

    if (_account.empty() || _endpoint_suffix.empty()) {
        return false;
    }

    if (slash_splits.size() > 1) {
        _container = slash_splits[1];

        if (slash_splits.size() > 2) {
            _blob_name = slash_splits[2];
        } else {
            _blob_name.clear();
        }
    } else {
        _container.clear();
        _blob_name.clear();
    }

    return true;
}
```
created_at: 2025-05-19T02:21:29Z
author_association: NONE

PR comments:
user: login: sonarqubecloud[bot]
created_at: 2025-05-20T07:36:20Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=StarRocks_starrocks&pullRequest=59061) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=StarRocks_starrocks&pullRequest=59061&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=StarRocks_starrocks&pullRequest=59061&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=StarRocks_starrocks&pullRequest=59061&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=StarRocks_starrocks&pullRequest=59061&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=StarRocks_starrocks&pullRequest=59061&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=StarRocks_starrocks&pullRequest=59061)


author_association: NONE
user: login: github-actions[bot]
created_at: 2025-05-21T20:33:08Z
body: #### [Java-Extensions Incremental Coverage Report]


:white_check_mark: **pass** : 0 / 0 (0%)




author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-21T20:33:10Z
body: #### [FE Incremental Coverage Report]


:white_check_mark: **pass** : 0 / 0 (0%)




author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-21T20:36:40Z
body: #### [BE Incremental Coverage Report]

:white_check_mark: **pass** : 84 / 91 (92.31%)




#### file detail

|   |path|covered_line|new_line|coverage|not_covered_line_detail|
|----|----|----|----|----|----|
|:large_blue_circle:|be/src/fs/fs.cpp|0|5|00.00%|[158, 159, 163, 164, 165]|
|:large_blue_circle:|be/src/fs/azure/utils.cpp|83|85|97.65%|[120, 121]|
|:large_blue_circle:|be/src/common/status.h|1|1|100.00%|[]|




author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-22T02:07:09Z
body: @Mergifyio backport branch-3.5
author_association: CONTRIBUTOR
user: login: mergify[bot]
created_at: 2025-05-22T02:07:18Z
body: > backport branch-3.5

#### ✅ Backports have been created

<details>

* [#59222 [Enhancement] Add azure blob file system in backend (backport #59061)](https://github.com/StarRocks/starrocks/pull/59222) has been created for branch `branch-3.5`

</details>


<!---
DO NOT EDIT
-*- Mergify Payload -*-
{"command": "backport branch-3.5", "conclusion": "success", "action_is_running": true}
-*- Mergify Payload End -*-
-->
author_association: CONTRIBUTOR