Repository: umccr/cloud-checksum
Pull request #52: feat: AWS credentials options
Author: login: mmalenic
Labels: name: enhancement
description: New feature or request
Created: 2025-05-09T04:50:43Z | Merged: 2025-05-12T06:36:49Z

Closes #48

I think this also maybe closes #50 as well?

### Changes
* Adds `Credentials` global options to specify different credentials for the copy source and destination.
    * The provider can be used to specify default credentials, no credentials or an AWS profile.
    * This adds source and destination parameters for the credentials-provider, region, endpoint-url and profile.
    * For `check` and `generate` only source credentials are considered, and destination parameters are ignored.
* If non-default credentials are used for source/destination, the copy mode is always `download-upload`. 
    * There might be some cases where `server-side` can be used even with different credentials, such as access point copies with different accounts/regions. Access point `copy_source` isn't implemented for now anyway, so I've just made it `download-upload` always.
* Improve compatibility with S3-compatiable storage.
    * Some credential combinations and endpoints don't support calls like `GetObjectAttributes`. Now, a call to `GetObjectAttributes` shows as a recoverable error in the `api_errors` stats fields and `HeadObject` is used if `GetObjectAttributes` fails.
* Improve error formatting and context for AWS errors.
    * The JSON output now properly formats the error code, message and the API call that caused it.
* Changes to `check`/`generate` task builders allowing individual s3 clients per input object.

Linked issue #48: Allow sourcing data from open data (anonymous) buckets
For test purposes (and for I guess more general purposes) - it is useful to be able to source data from AWS open data repositories.

To access an open data object - requests must be made with _no_ v4 signing. There is an option in AWS SDKs to enable this. I suggest the use of a flag indicating that a user wants no signing (for instance the aws cli uses `--no-sign-request` - but I suggest we integrate this with some sort of `--source-credential-type=None` (see issue #33))


Linked issue comments: user: login: andrewpatto
created_at: 2025-05-05T23:55:51Z
body: An example Sydney region object is `s3://threatenedspecies/index.html`

author_association: MEMBER
user: login: andrewpatto
created_at: 2025-05-06T00:08:14Z
body: PR #49 has some snippets of code that got this working - but will need some more extensive refactoring of the client construction before it was general purpose. So not suggesting using the PR - but can copy/paste from it.

author_association: MEMBER

Diff:
diff --git a/.gitignore b/.gitignore
index b3236f1..8b80227 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@ target/
 
 /.idea/
 .DS_Store
+.env
diff --git a/Cargo.lock b/Cargo.lock
index ea234ae..061e964 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -899,6 +899,7 @@ dependencies = [
  "crc32fast",
  "crc64fast-nvme",
  "criterion",
+ "dotenvy",
  "dyn-clone",
  "envy",
  "futures-util",
@@ -1168,6 +1169,12 @@ dependencies = [
  "syn 2.0.100",
 ]
 
+[[package]]
+name = "dotenvy"
+version = "0.15.7"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b"
+
 [[package]]
 name = "dunce"
 version = "1.0.5"
diff --git a/README.md b/README.md
index 9f6047b..f3d5372 100644
--- a/README.md
+++ b/README.md
@@ -82,4 +82,10 @@ using:
 CLOUD_CHECKSUM_TEST_BUCKET_URI="s3://bucket/prefix" cargo test --all-features -- --ignored
 ```
 
+The endpoint URL can also be set for S3-compatible endpoint tests:
+
+```sh
+CLOUD_CHECKSUM_TEST_BUCKET_URI="s3://bucket/prefix" CLOUD_CHECKSUM_TEST_ENDPOINT_URL="https://storage.googleapis.com" cargo test --all-features -- --ignored
+```
+
 
diff --git a/cloud-checksum/Cargo.toml b/cloud-checksum/Cargo.toml
index 8ac7a92..cb9a551 100644
--- a/cloud-checksum/Cargo.toml
+++ b/cloud-checksum/Cargo.toml
@@ -45,6 +45,7 @@ criterion = { version = "0.5", features = ["async_tokio", "html_reports"] }
 anyhow = "1"
 tempfile = "3"
 envy = "0.4"
+dotenvy = "0.15"
 
 aws-smithy-mocks-experimental = "0.2"
 
diff --git a/cloud-checksum/src/checksum/aws_etag.rs b/cloud-checksum/src/checksum/aws_etag.rs
index a1d06b7..e98fea4 100644
--- a/cloud-checksum/src/checksum/aws_etag.rs
+++ b/cloud-checksum/src/checksum/aws_etag.rs
@@ -3,7 +3,7 @@
 //!
 
 use crate::checksum::standard::StandardCtx;
-use crate::error::Error::{AwsError, ParseError};
+use crate::error::Error::ParseError;
 use crate::error::{Error, Result};
 use crate::io::Provider;
 use std::cmp::Ordering;
@@ -268,7 +268,7 @@ impl AWSETagCtx {
             let (data, remainder) = data.split_at(usize::try_from(
                 self.current_part_size
                     .checked_sub(self.current_bytes)
-                    .ok_or_else(|| AwsError("part size too large".to_string()))?,
+                    .ok_or_else(|| Error::aws_error("part size too large".to_string()))?,
             )?);
 
             self.ctx.update(Arc::from(data))?;
diff --git a/cloud-checksum/src/checksum/file.rs b/cloud-checksum/src/checksum/file.rs
index 4c79c53..b36e5fe 100644
--- a/cloud-checksum/src/checksum/file.rs
+++ b/cloud-checksum/src/checksum/file.rs
@@ -26,9 +26,10 @@ pub struct State {
 
 impl State {
     /// Build from a name.
-    pub async fn try_from(name: String) -> Result<Self> {
+    pub async fn try_from(name: String, avoid_get_object_attributes: bool) -> Result<Self> {
         Ok(Self {
             object_sums: ObjectSumsBuilder::default()
+                .with_avoid_get_object_attributes(avoid_get_object_attributes)
                 .build(SumsFile::format_target_file(&name))
                 .await?,
             name,
@@ -226,6 +227,11 @@ impl SumsFile {
     pub fn add_checksum(&mut self, ctx: Ctx, checksum: Checksum) {
         self.checksums.insert(ctx, checksum);
     }
+
+    /// Does the sums file contain no checksums.
+    pub fn is_empty(&self) -> bool {
+        self.checksums.is_empty()
+    }
 }
 
 impl TryFrom<&[u8]> for SumsFile {
diff --git a/cloud-checksum/src/cli.rs b/cloud-checksum/src/cli.rs
index abe9dc9..e596685 100644
--- a/cloud-checksum/src/cli.rs
+++ b/cloud-checksum/src/cli.rs
@@ -8,7 +8,7 @@ use crate::error::Error::{CheckError, ParseError};
 use crate::error::Result;
 use crate::io::sums::channel::ChannelReader;
 use crate::io::sums::ObjectSumsBuilder;
-use crate::io::{default_s3_client, Provider};
+use crate::io::{create_s3_client, default_s3_client, Provider};
 use crate::stats::{CheckStats, ChecksumPair, CopyStats, GenerateFileStats, GenerateStats};
 use crate::task::check::{CheckTask, CheckTaskBuilder, GroupBy};
 use crate::task::copy::CopyTaskBuilder;
@@ -19,6 +19,7 @@ use humantime::Duration;
 use parse_size::parse_size;
 use serde::{Deserialize, Serialize};
 use serde_json::{to_string, to_string_pretty};
+use std::collections::HashSet;
 use std::ffi::OsString;
 use std::fmt::{Display, Formatter};
 use std::str::FromStr;
@@ -43,6 +44,9 @@ pub struct Command {
     /// Options related to outputting data from the CLI.
     #[command(flatten)]
     pub output: Output,
+    /// Options related to credentials.
+    #[command(flatten)]
+    pub credentials: Credentials,
 }
 
 impl Command {
@@ -81,19 +85,30 @@ impl Command {
             }
         }
 
+        let credentials = &args.credentials;
+        if (credentials.source_credential_provider.is_aws() && credentials.source_profile.is_none())
+            || (credentials.destination_credential_provider.is_aws()
+                && credentials.destination_profile.is_none())
+        {
+            return Err(ParseError(
+                "a profile must be specified when using the `aws-profile` credential provider"
+                    .to_string(),
+            ));
+        }
+
         Ok(())
     }
 
     /// Execute the command from the args.
     pub async fn execute(self) -> Result<()> {
-        let client = Arc::new(default_s3_client().await?);
+        let client = Arc::new(self.credentials.source_client().await?);
 
         let pretty_json = self.output.pretty_json;
         let write_sums_file = self.output.write_sums_file;
         match self.commands {
             Subcommands::Generate(generate_args) => {
                 let (sums, stats) = generate_args
-                    .generate(self.optimization, client, true)
+                    .generate(self.optimization, &self.credentials, vec![client], true)
                     .await
                     .inspect_err(|err| {
                         Self::print_stats(err, pretty_json).ok();
@@ -107,20 +122,36 @@ impl Command {
             }
             Subcommands::Check(check_args) => {
                 let output = check_args
-                    .check(self.optimization, client, write_sums_file, false)
+                    .check(
+                        self.optimization,
+                        &self.credentials,
+                        write_sums_file,
+                        false,
+                        vec![client],
+                    )
                     .await
                     .inspect_err(|err| {
                         Self::print_stats(err, pretty_json).ok();
                     })?;
+
                 Self::print_stats(&output, pretty_json)?;
             }
             Subcommands::Copy(copy_args) => {
+                let destination_client = Arc::new(self.credentials.destination_client().await?);
+
                 let output = copy_args
-                    .copy(client, self.optimization, write_sums_file)
+                    .copy(
+                        client,
+                        destination_client,
+                        self.credentials,
+                        self.optimization,
+                        write_sums_file,
+                    )
                     .await
                     .inspect_err(|err| {
                         Self::print_stats(err, pretty_json).ok();
                     })?;
+
                 Self::print_stats(&output, pretty_json)?;
             }
         }
@@ -175,8 +206,7 @@ pub struct Generate {
     pub checksum: Vec<Ctx>,
     /// Generate any missing checksums that would be required to confirm whether two files are
     /// identical using the `check` subcommand. Any additional checksums specified using
-    /// `--checksum` will also be generated. If there are no checksums preset, the default
-    /// checksum is generated.
+    /// `--checksum` will also be generated.
     #[arg(short, long, env)]
     pub missing: bool,
     /// Overwrite the sums file. By default, only checksums that are missing are computed and
@@ -199,18 +229,20 @@ impl Generate {
     pub async fn generate(
         self,
         optimization: Optimization,
-        client: Arc<Client>,
+        credentials: &Credentials,
+        mut clients: Vec<Arc<Client>>,
         write_sums_file: bool,
-    ) -> Result<(Vec<SumsFile>, Option<GenerateStats>)> {
+    ) -> Result<(Vec<(String, SumsFile)>, Option<GenerateStats>)> {
         if self.input[0] == "-" {
             let reader = ChannelReader::new(stdin(), optimization.channel_capacity);
 
             let output = GenerateTaskBuilder::default()
+                .with_avoid_get_object_attributes(credentials.avoid_get_object_attributes)
                 .with_overwrite(self.force_overwrite)
                 .with_verify(self.verify)
                 .with_context(self.checksum)
                 .with_reader(reader)
-                .with_client(client)
+                .set_client(clients.first().cloned())
                 .build()
                 .await?
                 .run()
@@ -218,18 +250,23 @@ impl Generate {
                 .into_inner()
                 .0;
 
-            Ok((vec![output], None))
+            Ok((vec![(self.input[0].to_string(), output)], None))
         } else {
             let now = Instant::now();
             let mut check_stats = None;
             let mut generate_stats = vec![];
             let mut sums_files = vec![];
+            let mut errors = HashSet::new();
 
             if self.missing {
                 let now = Instant::now();
-                let (ctxs, group_by) =
-                    Check::comparable_check(self.input.clone(), client.clone()).await?;
-                let (objects, compared, updated) = ctxs.into_inner();
+                let (ctxs, group_by) = Check::comparable_check(
+                    self.input.clone(),
+                    clients.clone(),
+                    credentials.avoid_get_object_attributes,
+                )
+                .await?;
+                let (objects, compared, updated, api_errors) = ctxs.into_inner();
                 check_stats = Some(CheckStats::new(
                     now.elapsed().as_secs_f64(),
                     group_by,
@@ -237,45 +274,72 @@ impl Generate {
                     objects.to_groups(),
                     updated,
                     None,
+                    api_errors,
                 ));
 
+                if clients.is_empty() {
+                    clients = vec![Arc::new(default_s3_client().await?)];
+                }
+
                 let ctxs = SumCtxPairs::from_comparable(objects)?;
                 if let Some(ctxs) = ctxs {
-                    for ctx in ctxs.into_inner() {
+                    for (ctx, client) in ctxs
+                        .into_inner()
+                        .into_iter()
+                        .zip(clients.clone().into_iter().cycle())
+                    {
                         let (input, ctx) = ctx.into_inner();
                         let task = GenerateTaskBuilder::default()
+                            .with_avoid_get_object_attributes(
+                                credentials.avoid_get_object_attributes,
+                            )
                             .with_overwrite(self.force_overwrite)
                             .with_verify(self.verify)
-                            .with_input_file_name(input)
+                            .with_input_file_name(input.to_string())
                             .with_context(vec![ctx])
                             .with_capacity(optimization.channel_capacity)
-                            .with_client(client.clone())
+                            .with_client(client)
                             .set_write(write_sums_file)
                             .build()
                             .await?
                             .run()
                             .await?;
 
-                        sums_files.push(task.sums_file().clone());
+                        sums_files.push((input, task.sums_file().clone()));
+                        errors.extend(task.api_errors());
                         generate_stats.push(GenerateFileStats::from_task(task));
                     }
                 }
+
+                if self.checksum.is_empty() {
+                    return Ok((
+                        sums_files,
+                        Some(GenerateStats::new(
+                            now.elapsed().as_secs_f64(),
+                            generate_stats,
+                            check_stats,
+                            errors,
+                        )),
+                    ));
+                }
             };
 
-            for input in self.input {
+            for (input, client) in self.input.into_iter().zip(clients.into_iter().cycle()) {
                 let task = GenerateTaskBuilder::default()
+                    .with_avoid_get_object_attributes(credentials.avoid_get_object_attributes)
                     .with_overwrite(self.force_overwrite)
                     .with_verify(self.verify)
-                    .with_input_file_name(input)
+                    .with_input_file_name(input.to_string())
                     .with_context(self.checksum.clone())
                     .with_capacity(optimization.channel_capacity)
-                    .with_client(client.clone())
+                    .with_client(client)
                     .set_write(write_sums_file)
                     .build()
                     .await?
                     .run()
                     .await?;
-                sums_files.push(task.sums_file().clone());
+                sums_files.push((input, task.sums_file().clone()));
+                errors.extend(task.api_errors());
                 generate_stats.push(GenerateFileStats::from_task(task));
             }
 
@@ -285,6 +349,7 @@ impl Generate {
                     now.elapsed().as_secs_f64(),
                     generate_stats,
                     check_stats,
+                    errors,
                 )),
             ))
         }
@@ -317,13 +382,15 @@ impl Check {
     /// Perform a check for comparability on the input files.
     pub async fn comparable_check(
         input: Vec<String>,
-        client: Arc<Client>,
+        clients: Vec<Arc<Client>>,
+        avoid_get_object_attributes: bool,
     ) -> Result<(CheckTask, GroupBy)> {
         Ok((
             CheckTaskBuilder::default()
                 .with_input_files(input)
                 .with_group_by(GroupBy::Comparability)
-                .with_client(client)
+                .with_avoid_get_object_attributes(avoid_get_object_attributes)
+                .with_clients(clients)
                 .build()
                 .await?
                 .run()
@@ -334,7 +401,7 @@ impl Check {
 
     /// Determine sums to generate based on a comparability check
     fn generate_sums(ctxs: CheckTask) -> Vec<Ctx> {
-        if ctxs.compared_directly().is_empty() {
+        if ctxs.is_empty() {
             vec![Ctx::default()]
         } else {
             vec![]
@@ -345,20 +412,28 @@ impl Check {
     pub async fn check(
         self,
         optimization: Optimization,
-        client: Arc<Client>,
+        credentials: &Credentials,
         write_sums_file: bool,
         verify: bool,
+        clients: Vec<Arc<Client>>,
     ) -> Result<CheckStats> {
         let now = Instant::now();
         let group_by = self.group_by;
 
-        let builder = CheckTaskBuilder::default()
+        let mut builder = CheckTaskBuilder::default()
             .with_group_by(group_by)
+            .with_avoid_get_object_attributes(credentials.avoid_get_object_attributes)
+            .with_input_files(self.input.clone())
             .with_update(self.update)
-            .with_client(client.clone());
+            .with_clients(clients.clone());
         let mut generate_stats = None;
-        let builder = if self.missing {
-            let (ctxs, _) = Check::comparable_check(self.input.clone(), client.clone()).await?;
+        if self.missing {
+            let (ctxs, _) = Check::comparable_check(
+                self.input.clone(),
+                clients.clone(),
+                credentials.avoid_get_object_attributes,
+            )
+            .await?;
             let checksum = Check::generate_sums(ctxs);
 
             let (sums, stats) = Generate {
@@ -368,14 +443,12 @@ impl Check {
                 force_overwrite: false,
                 verify,
             }
-            .generate(optimization, client.clone(), write_sums_file)
+            .generate(optimization, credentials, clients.clone(), write_sums_file)
             .await?;
             generate_stats = stats;
 
-            builder.with_sums_files(self.input.into_iter().zip(sums).collect())
-        } else {
-            builder.with_input_files(self.input)
-        };
+            builder = builder.with_sums_files(sums);
+        }
 
         let check = builder.build().await?.run().await?;
         if check.compared_directly().is_empty() {
@@ -433,6 +506,40 @@ impl CopyMode {
     pub fn is_download_upload(&self) -> bool {
         matches!(self, CopyMode::DownloadUpload)
     }
+
+    /// Is this a server-side copy operation.
+    pub fn is_server_side(&self) -> bool {
+        matches!(self, CopyMode::ServerSide)
+    }
+}
+
+/// Details of how to locate credentials or specify no credentials needed
+#[derive(Debug, Clone, ValueEnum, Copy, Default, Deserialize, Serialize)]
+pub enum CredentialProvider {
+    /// Use the default mechanism of the SDK that obtains credentials from the system
+    #[default]
+    DefaultEnvironment,
+    /// Explicitly state that we want to attempt operations using no credentials (no SDK signing)
+    NoCredentials,
+    /// An AWS profile name.
+    AwsProfile,
+}
+
+impl CredentialProvider {
+    /// Is this source of credentials one with no credentials (i.e. anonymous unsigned calls)
+    pub fn is_anonymous(&self) -> bool {
+        matches!(self, CredentialProvider::NoCredentials)
+    }
+
+    /// Is this an aws-profile credential provider.
+    pub fn is_aws(&self) -> bool {
+        matches!(self, CredentialProvider::AwsProfile)
+    }
+
+    /// Is this a default credential provider.
+    pub fn is_default(&self) -> bool {
+        matches!(self, CredentialProvider::DefaultEnvironment)
+    }
 }
 
 /// The copy subcommand components.
@@ -485,8 +592,10 @@ pub struct Copy {
 impl Copy {
     pub async fn copy_check(
         &self,
-        client: Arc<Client>,
+        source_client: Arc<Client>,
+        destination_client: Arc<Client>,
         optimization: Optimization,
+        credentials: &Credentials,
         verify: bool,
         write_sums_file: bool,
     ) -> Result<CheckStats> {
@@ -498,7 +607,13 @@ impl Copy {
             group_by: GroupBy::Equality,
             missing: true,
         }
-        .check(optimization, client, write_sums_file, verify)
+        .check(
+            optimization,
+            credentials,
+            write_sums_file,
+            verify,
+            vec![source_client, destination_client],
+        )
         .await?;
 
         Ok(result)
@@ -507,7 +622,9 @@ impl Copy {
     /// Perform the copy sub command from the args.
     pub async fn copy(
         self,
-        client: Arc<Client>,
+        source_client: Arc<Client>,
+        destination_client: Arc<Client>,
+        credentials: Credentials,
         optimization: Optimization,
         write_sums_file: bool,
     ) -> Result<CopyStats> {
@@ -517,7 +634,8 @@ impl Copy {
         if !self.no_skip {
             // Check if it exists in the first place.
             let file_size = ObjectSumsBuilder::default()
-                .set_client(Some(client.clone()))
+                .set_client(Some(source_client.clone()))
+                .with_avoid_get_object_attributes(credentials.avoid_get_object_attributes)
                 .build(self.destination.to_string())
                 .await?
                 .file_size()
@@ -529,7 +647,14 @@ impl Copy {
 
             if exists {
                 let check_stats = self
-                    .copy_check(client.clone(), optimization.clone(), false, write_sums_file)
+                    .copy_check(
+                        source_client.clone(),
+                        destination_client.clone(),
+                        optimization.clone(),
+                        &credentials,
+                        false,
+                        write_sums_file,
+                    )
                     .await?;
 
                 if check_stats.groups.len() == 1 {
@@ -543,7 +668,7 @@ impl Copy {
                         skipped: true,
                         sums_mismatch: false,
                         n_retries: 0,
-                        api_errors: vec![],
+                        api_errors: HashSet::new(),
                         check_stats: Some(check_stats),
                     };
                     return Ok(copy_stats);
@@ -551,15 +676,25 @@ impl Copy {
             }
         }
 
+        // The copy mode must be download-upload if not using default credential providers.
+        let copy_mode = if credentials.is_default() {
+            self.copy_mode
+        } else {
+            CopyMode::DownloadUpload
+        };
+
         let result = CopyTaskBuilder::default()
             .with_source(self.source.to_string())
             .with_destination(self.destination.to_string())
             .with_metadata_mode(self.metadata_mode)
+            .with_tag_mode(self.tag_mode)
             .with_multipart_threshold(self.multipart_threshold)
+            .with_avoid_get_object_attributes(credentials.avoid_get_object_attributes)
             .with_concurrency(self.concurrency)
             .with_part_size(self.part_size)
-            .with_copy_mode(self.copy_mode)
-            .with_client(client.clone())
+            .with_copy_mode(copy_mode)
+            .with_source_client(source_client.clone())
+            .with_destination_client(destination_client.clone())
             .build()
             .await?
             .run()
@@ -569,7 +704,14 @@ impl Copy {
         let sums_mismatch = exists;
         let copy_stats = if !self.no_check {
             let check_stats = self
-                .copy_check(client, optimization, sums_mismatch, write_sums_file)
+                .copy_check(
+                    source_client,
+                    destination_client,
+                    optimization,
+                    &credentials,
+                    sums_mismatch,
+                    write_sums_file,
+                )
                 .await?;
             CopyStats::from_task(
                 result,
@@ -672,3 +814,84 @@ pub struct Output {
     #[arg(global = true, long, env)]
     pub write_sums_file: bool,
 }
+
+/// Options related to credentials. Options prefixed with `source_` affect `check`, `generate` and
+/// the source of a `copy` command. These options also have an alias without the prefix as they are
+/// used in all commands. Options prefixed with `destination_` only affect the destination of a
+/// `copy` command.
+#[derive(Args, Debug)]
+#[group(required = false)]
+pub struct Credentials {
+    /// The credentials source credentials to use. This affects the credentials used for `check`
+    /// `generate` and the source of a `copy` operation.
+    #[arg(
+        global = true,
+        long,
+        env,
+        default_value = "default-environment",
+        alias = "credential-provider"
+    )]
+    pub source_credential_provider: CredentialProvider,
+    /// The destination credentials to use. This only affects the credentials used for the
+    /// destination of a `copy` operation.
+    #[arg(global = true, long, env, default_value = "default-environment")]
+    pub destination_credential_provider: CredentialProvider,
+    /// The source profile to use if the source credential provider is `aws-profile`.
+    /// This must be specified if using `aws-profile`.
+    #[arg(global = true, long, env, alias = "profile")]
+    pub source_profile: Option<String>,
+    /// The destination profile to use if the destination credential provider is `aws-profile`.
+    /// This must be specified if using `aws-profile`.
+    #[arg(global = true, long, env)]
+    pub destination_profile: Option<String>,
+    /// Set the region for the source credential provider.
+    #[arg(global = true, long, env, alias = "region")]
+    pub source_region: Option<String>,
+    /// Set the region for the source credential provider.
+    #[arg(global = true, long, env)]
+    pub destination_region: Option<String>,
+    /// Set the source endpoint URL for AWS calls. This allows using a different endpoint that
+    /// has an S3-compatible storage API.
+    #[arg(global = true, long, env, alias = "endpoint-url")]
+    pub source_endpoint_url: Option<String>,
+    /// Set the destination endpoint URL for AWS calls. This allows using a different endpoint
+    /// that has an S3-compatible storage API.
+    #[arg(global = true, long, env)]
+    pub destination_endpoint_url: Option<String>,
+    /// Avoid `GetObjectAttributes` calls when determining sums. `HeadObject` will be used as a
+    /// fallback. `GetObjectAttributes` is preferred over `HeadObject` because it only requires
+    /// a single call rather than a call for each part.
+    #[arg(global = true, long, env)]
+    pub avoid_get_object_attributes: bool,
+}
+
+impl Credentials {
+    /// Construct the source client from the credentials.
+    pub async fn source_client(&self) -> Result<Client> {
+        create_s3_client(
+            &self.source_credential_provider,
+            self.source_profile.as_deref(),
+            self.source_region.as_deref(),
+            self.source_endpoint_url.as_deref(),
+        )
+        .await
+    }
+
+    /// Construct the destination client from the credentials.
+    pub async fn destination_client(&self) -> Result<Client> {
+        create_s3_client(
+            &self.destination_credential_provider,
+            self.destination_profile.as_deref(),
+            self.destination_region.as_deref(),
+            self.destination_endpoint_url.as_deref(),
+        )
+        .await
+    }
+
+    pub fn is_default(&self) -> bool {
+        self.source_credential_provider.is_default()
+            && self.destination_credential_provider.is_default()
+            && self.source_endpoint_url.is_none()
+            && self.destination_endpoint_url.is_none()
+    }
+}
diff --git a/cloud-checksum/src/error.rs b/cloud-checksum/src/error.rs
index 62b7f85..27b90f0 100644
--- a/cloud-checksum/src/error.rs
+++ b/cloud-checksum/src/error.rs
@@ -1,12 +1,27 @@
 //! Error handling logic.
 //!
 
+use crate::error::Error::AwsError;
 use aws_sdk_s3::error::SdkError;
+use aws_sdk_s3::operation::complete_multipart_upload::CompleteMultipartUploadError;
+use aws_sdk_s3::operation::copy_object::CopyObjectError;
+use aws_sdk_s3::operation::create_multipart_upload::CreateMultipartUploadError;
+use aws_sdk_s3::operation::get_object::GetObjectError;
+use aws_sdk_s3::operation::get_object_attributes::GetObjectAttributesError;
+use aws_sdk_s3::operation::get_object_tagging::GetObjectTaggingError;
+use aws_sdk_s3::operation::head_object::HeadObjectError;
+use aws_sdk_s3::operation::put_object::PutObjectError;
+use aws_sdk_s3::operation::upload_part::UploadPartError;
+use aws_sdk_s3::operation::upload_part_copy::UploadPartCopyError;
+use aws_smithy_runtime_api::client::orchestrator::HttpResponse;
 use aws_smithy_runtime_api::client::result::CreateUnhandledError;
 use aws_smithy_types::byte_stream;
+use aws_smithy_types::error::display::DisplayErrorContext;
+use aws_smithy_types::error::metadata::ProvideErrorMetadata;
 use serde::{Deserialize, Serialize, Serializer};
+use std::fmt::{Debug, Display, Formatter};
 use std::num::TryFromIntError;
-use std::{error, io, result};
+use std::{error, fmt, io, result};
 use thiserror::Error;
 use tokio::sync::mpsc;
 use tokio::task::JoinError;
@@ -15,7 +30,7 @@ use tokio::task::JoinError;
 pub type Result<T> = result::Result<T, Error>;
 
 /// Error types for checksum_cloud.
-#[derive(Error, Debug, Serialize, Deserialize)]
+#[derive(Error, Serialize, Deserialize)]
 pub enum Error {
     #[error("in concurrency logic: {0}")]
     ConcurrencyError(String),
@@ -39,8 +54,43 @@ pub enum Error {
     CheckError(String),
     #[error("copy command error: {0}")]
     CopyError(String),
-    #[error("aws error: {0}")]
-    AwsError(String),
+    #[serde(serialize_with = "serialize_aws_error")]
+    #[error("aws error: {message}")]
+    AwsError {
+        message: String,
+        api_error: Option<ApiError>,
+    },
+}
+
+impl Debug for Error {
+    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
+        write!(f, "{}", self)
+    }
+}
+
+impl Error {
+    /// Create an AWS error from a string.
+    pub fn aws_error(err: String) -> Self {
+        AwsError {
+            message: err.to_string(),
+            api_error: None,
+        }
+    }
+}
+
+fn serialize_aws_error<S>(
+    err: &str,
+    api_error: &Option<ApiError>,
+    serializer: S,
+) -> result::Result<S::Ok, S::Error>
+where
+    S: Serializer,
+{
+    if let Some(api_error) = api_error {
+        api_error.serialize(serializer)
+    } else {
+        err.serialize(serializer)
+    }
 }
 
 fn serialize_try_from_int<S>(
@@ -84,17 +134,97 @@ impl From<serde_json::Error> for Error {
     }
 }
 
-impl<T> From<SdkError<T>> for Error
+impl From<byte_stream::error::Error> for Error {
+    fn from(err: byte_stream::error::Error) -> Self {
+        Self::IOError(io::Error::other(err))
+    }
+}
+
+/// An API error that could be returned from storage.
+#[derive(Serialize, Deserialize, Debug, Clone, Eq, PartialOrd, PartialEq, Ord, Hash)]
+pub struct ApiError {
+    /// The error kind, e.g. `AccessDenied`.
+    pub(crate) code: String,
+    /// The API call.
+    pub(crate) call: String,
+    /// The error message.
+    pub(crate) message: String,
+}
+
+impl ApiError {
+    /// Create a new error.
+    pub fn new(code: String, call: String, message: String) -> Self {
+        Self {
+            code,
+            call,
+            message,
+        }
+    }
+
+    /// Check if the error is an access denied error.
+    pub fn is_access_denied(&self) -> bool {
+        self.code == "AccessDenied"
+    }
+}
+
+impl<T> From<(&SdkError<T, HttpResponse>, String)> for ApiError
 where
-    T: Send + Sync + error::Error + CreateUnhandledError + 'static,
+    T: ProvideErrorMetadata + CreateUnhandledError + error::Error + Send + Sync + 'static,
 {
-    fn from(err: SdkError<T>) -> Self {
-        Self::AwsError(err.into_service_error().to_string())
+    fn from((err, call): (&SdkError<T, HttpResponse>, String)) -> Self {
+        Self::new(
+            err.code().unwrap_or("Unknown").to_string(),
+            call,
+            err.message()
+                .map(|msg| msg.to_string())
+                .or_else(|| err.as_service_error().map(|err| err.to_string()))
+                .unwrap_or_else(|| DisplayErrorContext(&err).to_string()),
+        )
     }
 }
 
-impl From<byte_stream::error::Error> for Error {
-    fn from(err: byte_stream::error::Error) -> Self {
-        Self::AwsError(err.to_string())
+impl Display for ApiError {
+    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
+        write!(f, "{} for {}: {}", self.code, self.call, self.message)
     }
 }
+
+/// Generate an impl for an AWS error type with the context of the API call.
+macro_rules! generate_aws_error_impl {
+    ($t:ty) => {
+        impl From<&SdkError<$t>> for ApiError {
+            fn from(err: &SdkError<$t>) -> Self {
+                let api_call = stringify!($t);
+                (
+                    err,
+                    api_call
+                        .strip_suffix("Error")
+                        .unwrap_or(api_call)
+                        .to_string(),
+                )
+                    .into()
+            }
+        }
+
+        impl From<SdkError<$t>> for Error {
+            fn from(err: SdkError<$t>) -> Self {
+                let err = ApiError::from(&err);
+                Self::AwsError {
+                    message: err.to_string(),
+                    api_error: Some(err),
+                }
+            }
+        }
+    };
+}
+
+generate_aws_error_impl!(HeadObjectError);
+generate_aws_error_impl!(GetObjectAttributesError);
+generate_aws_error_impl!(PutObjectError);
+generate_aws_error_impl!(GetObjectTaggingError);
+generate_aws_error_impl!(CreateMultipartUploadError);
+generate_aws_error_impl!(CompleteMultipartUploadError);
+generate_aws_error_impl!(CopyObjectError);
+generate_aws_error_impl!(UploadPartCopyError);
+generate_aws_error_impl!(GetObjectError);
+generate_aws_error_impl!(UploadPartError);
diff --git a/cloud-checksum/src/io/copy/aws.rs b/cloud-checksum/src/io/copy/aws.rs
index 3928b6e..18ca279 100644
--- a/cloud-checksum/src/io/copy/aws.rs
+++ b/cloud-checksum/src/io/copy/aws.rs
@@ -3,10 +3,9 @@
 
 use crate::checksum::file::SumsFile;
 use crate::cli::MetadataCopy;
-use crate::error::Error::{AwsError, CopyError, ParseError};
-use crate::error::{Error, Result};
+use crate::error::Error::{CopyError, ParseError};
+use crate::error::{ApiError, Error, Result};
 use crate::io::copy::{CopyContent, CopyResult, CopyState, MultiPartOptions, ObjectCopy, Part};
-use crate::stats::ApiError;
 use aws_sdk_s3::operation::get_object_tagging::{GetObjectTaggingError, GetObjectTaggingOutput};
 use aws_sdk_s3::operation::head_object::{HeadObjectError, HeadObjectOutput};
 use aws_sdk_s3::operation::upload_part::UploadPartOutput;
@@ -18,8 +17,6 @@ use aws_sdk_s3::Client;
 use aws_smithy_runtime_api::client::orchestrator::HttpResponse;
 use aws_smithy_runtime_api::client::result::SdkError;
 use aws_smithy_types::byte_stream::ByteStream;
-use aws_smithy_types::error::display::DisplayErrorContext;
-use aws_smithy_types::error::metadata::ProvideErrorMetadata;
 use std::collections::HashMap;
 use std::result;
 use std::sync::Arc;
@@ -189,25 +186,24 @@ impl S3 {
         let tags = self.tagging(&key, &bucket).await;
 
         // Getting tags could fail, that's okay if using best-effort mode.
-        let tags =
-            if self.tag_mode.is_best_effort() && tags.as_ref().is_err_and(Self::is_access_denied) {
-                None
-            } else {
-                Some(
-                    tags?
-                        .tag_set
-                        .iter()
-                        .map(|tag| format!("{}={}", tag.key(), tag.value()))
-                        .collect::<Vec<_>>()
-                        .join("&"),
-                )
-            };
+        let tags = if self.tag_mode.is_best_effort() {
+            None
+        } else {
+            Some(
+                tags?
+                    .tag_set
+                    .iter()
+                    .map(|tag| format!("{}={}", tag.key(), tag.value()))
+                    .collect::<Vec<_>>()
+                    .join("&"),
+            )
+        };
 
         let size = head
             .content_length
             .map(u64::try_from)
             .transpose()?
-            .ok_or_else(|| AwsError("missing size".to_string()))?;
+            .ok_or_else(|| Error::aws_error("missing size".to_string()))?;
         let metadata = head.metadata;
 
         Ok(CopyState::new(size, tags, metadata))
@@ -258,26 +254,6 @@ impl S3 {
         }
     }
 
-    /// Check if the error is an access denied error.
-    fn is_access_denied<T: ProvideErrorMetadata>(err: &SdkError<T, HttpResponse>) -> bool {
-        if let Some(err) = err.as_service_error() {
-            if err.code().is_some_and(|code| code == "AccessDenied") {
-                return true;
-            }
-        }
-
-        false
-    }
-
-    fn api_error<T: ProvideErrorMetadata + std::error::Error + 'static>(
-        err: &SdkError<T, HttpResponse>,
-    ) -> ApiError {
-        ApiError::new(
-            err.code().unwrap_or_default().to_string(),
-            DisplayErrorContext(err).to_string(),
-        )
-    }
-
     /// Create a new multipart upload.
     pub async fn get_multipart_upload(
         &self,
@@ -308,10 +284,11 @@ impl S3 {
 
         // Retry if this is a best effort copy and the error was access denied.
         let (upload, err) = if let Err(ref err) = result {
-            if self.tag_mode.is_best_effort() && Self::is_access_denied(err) {
+            let err = ApiError::from(err);
+            if self.tag_mode.is_best_effort() && err.is_access_denied() {
                 (
                     do_upload(None, metadata, additional_checksum).await?,
-                    vec![Self::api_error(err)],
+                    vec![err],
                 )
             } else {
                 (result?, vec![])
@@ -323,7 +300,7 @@ impl S3 {
         Ok((
             upload
                 .upload_id
-                .ok_or_else(|| AwsError("missing upload id".to_string()))?,
+                .ok_or_else(|| Error::aws_error("missing upload id".to_string()))?,
             err,
         ))
     }
@@ -377,7 +354,8 @@ impl S3 {
 
         // Retry if this is a best effort copy and the error was access denied.
         let (_, err) = if let Err(ref err) = result {
-            if self.tag_mode.is_best_effort() && Self::is_access_denied(err) {
+            let err = ApiError::from(err);
+            if self.tag_mode.is_best_effort() && err.is_access_denied() {
                 let result = do_copy(
                     TaggingDirective::Replace,
                     Some("".to_string()),
@@ -386,7 +364,7 @@ impl S3 {
                     additional_checksum,
                 )
                 .await?;
-                (result, vec![Self::api_error(err)])
+                (result, vec![err])
             } else {
                 (result?, vec![])
             }
@@ -463,12 +441,12 @@ impl S3 {
                 .copy_source_range(
                     multi_part
                         .format_range()
-                        .ok_or_else(|| AwsError("invalid range".to_string()))?,
+                        .ok_or_else(|| Error::aws_error("invalid range".to_string()))?,
                 )
                 .send()
                 .await?
                 .copy_part_result
-                .ok_or_else(|| AwsError("missing copy part result".to_string()))?;
+                .ok_or_else(|| Error::aws_error("missing copy part result".to_string()))?;
 
             let mut result: CopyResult = (part, part_number, upload_id).into();
             result.bytes_transferred = multi_part.bytes_transferred();
@@ -503,11 +481,10 @@ impl S3 {
             .get_object()
             .bucket(&source.bucket)
             .key(&source.key)
-            .set_part_number(
+            .set_range(
                 multi_part
                     .as_ref()
-                    .and_then(|multi_part| multi_part.part_number.map(i32::try_from))
-                    .transpose()?,
+                    .and_then(|multi_part| multi_part.format_range()),
             )
             .send()
             .await?;
@@ -522,7 +499,7 @@ impl S3 {
         state: &CopyState,
     ) -> Result<CopyResult> {
         let destination = self.get_destination()?;
-        let buf = Self::read_content(&mut content).await?;
+        let buf = Self::read_content(&mut content, None).await?;
 
         let additional_checksum = state.additional_ctx().map(ChecksumAlgorithm::from);
         let do_put = |tags, metadata, additional_checksum, buf| async {
@@ -548,10 +525,11 @@ impl S3 {
 
         // Retry if this is a best effort copy and the error was access denied.
         let (_, err) = if let Err(ref err) = result {
-            if self.tag_mode.is_best_effort() && Self::is_access_denied(err) {
+            let err = ApiError::from(err);
+            if self.tag_mode.is_best_effort() && err.is_access_denied() {
                 let result = do_put(None, state.metadata(), additional_checksum, buf).await?;
 
-                (result, vec![Self::api_error(err)])
+                (result, vec![err])
             } else {
                 (result?, vec![])
             }
@@ -563,12 +541,25 @@ impl S3 {
     }
 
     /// Read the copy content into a buffer.
-    async fn read_content(content: &mut CopyContent) -> Result<Vec<u8>> {
-        let mut buf = vec![];
+    async fn read_content(
+        content: &mut CopyContent,
+        multi_part: Option<&MultiPartOptions>,
+    ) -> Result<Vec<u8>> {
+        if let Some(multi_part) = multi_part {
+            if multi_part.part_number.is_none() {
+                return Ok(Vec::new());
+            }
+
+            let mut buf = vec![0; usize::try_from(multi_part.bytes_transferred())?];
+            content.data.read_exact(&mut buf).await?;
 
-        content.data.read_to_end(&mut buf).await?;
+            Ok(buf)
+        } else {
+            let mut buf = vec![];
+            content.data.read_to_end(&mut buf).await?;
 
-        Ok(buf)
+            Ok(buf)
+        }
     }
 
     /// Upload objects using multi part uploads.
@@ -579,7 +570,7 @@ impl S3 {
         state: &CopyState,
     ) -> Result<CopyResult> {
         let destination = self.get_destination()?;
-        let buf = Self::read_content(&mut content).await?;
+        let buf = Self::read_content(&mut content, Some(&multi_part)).await?;
 
         let additional_checksum = state.additional_ctx().map(ChecksumAlgorithm::from);
         // Create the upload id if it doesn't exist or use the existing one.
diff --git a/cloud-checksum/src/io/copy/file.rs b/cloud-checksum/src/io/copy/file.rs
index 130e060..91ec5b3 100644
--- a/cloud-checksum/src/io/copy/file.rs
+++ b/cloud-checksum/src/io/copy/file.rs
@@ -92,7 +92,11 @@ impl File {
     }
 
     /// Write the data to the destination.
-    pub async fn write(&self, mut data: CopyContent) -> Result<u64> {
+    pub async fn write(
+        &self,
+        mut data: CopyContent,
+        multipart: Option<MultiPartOptions>,
+    ) -> Result<u64> {
         let destination = self.get_destination()?;
         // Append to an existing file or create a new one.
         let mut file = if fs::try_exists(&destination)
@@ -108,7 +112,19 @@ impl File {
             fs::File::create(destination).await?
         };
 
-        let total = io::copy(&mut data.data, &mut file).await?;
+        let total = if let Some(multipart) = multipart {
+            if multipart.part_number.is_none() {
+                return Ok(0);
+            }
+
+            io::copy(
+                &mut data.data.take(multipart.bytes_transferred()),
+                &mut file,
+            )
+            .await?
+        } else {
+            io::copy(&mut data.data, &mut file).await?
+        };
 
         Ok(total)
     }
@@ -163,12 +179,12 @@ impl ObjectCopy for File {
     async fn upload(
         &self,
         data: CopyContent,
-        _multipart: Option<MultiPartOptions>,
+        multipart: Option<MultiPartOptions>,
         _state: &CopyState,
     ) -> Result<CopyResult> {
         // It doesn't matter what the part number is for filesystem operations, just append to the
         // end of the file as we assume correct ordering of parts.
-        let bytes = self.write(data).await?;
+        let bytes = self.write(data, multipart).await?;
 
         CopyResult::new(None, None, bytes, vec![])
     }
diff --git a/cloud-checksum/src/io/copy/mod.rs b/cloud-checksum/src/io/copy/mod.rs
index 57e1b77..0cf88b1 100644
--- a/cloud-checksum/src/io/copy/mod.rs
+++ b/cloud-checksum/src/io/copy/mod.rs
@@ -4,11 +4,10 @@
 use crate::checksum::Ctx;
 use crate::cli::MetadataCopy;
 use crate::error::Error::CopyError;
-use crate::error::Result;
+use crate::error::{ApiError, Result};
 use crate::io::copy::aws::S3Builder;
 use crate::io::copy::file::FileBuilder;
 use crate::io::{default_s3_client, Provider};
-use crate::stats::ApiError;
 use aws_sdk_s3::Client;
 use dyn_clone::DynClone;
 use std::collections::HashMap;
diff --git a/cloud-checksum/src/io/mod.rs b/cloud-checksum/src/io/mod.rs
index d248dbf..7e73ff0 100644
--- a/cloud-checksum/src/io/mod.rs
+++ b/cloud-checksum/src/io/mod.rs
@@ -1,10 +1,11 @@
 //! Module that handles all file IO
 //!
 
+use crate::cli::CredentialProvider;
 use crate::error::Error::ParseError;
 use crate::error::{Error, Result};
-use aws_config::load_defaults;
-use aws_sdk_s3::Client;
+use aws_config::Region;
+use aws_sdk_s3::{config, Client};
 use aws_smithy_runtime_api::client::behavior_version::BehaviorVersion;
 
 pub mod copy;
@@ -106,9 +107,41 @@ impl TryFrom<&str> for Provider {
     }
 }
 
+/// Create an S3 client from the credentials provider, profile, region and endpoint url.
+pub async fn create_s3_client(
+    provider: &CredentialProvider,
+    profile: Option<&str>,
+    region: Option<&str>,
+    endpoint_url: Option<&str>,
+) -> Result<Client> {
+    let mut loader = aws_config::defaults(BehaviorVersion::latest());
+
+    if let Some(region) = region {
+        loader = loader.region(Region::new(region.to_string()));
+    }
+    if let Some(endpoint_url) = endpoint_url {
+        loader = loader.endpoint_url(endpoint_url);
+    }
+
+    let loader = match (provider, profile) {
+        (CredentialProvider::DefaultEnvironment, _) => loader,
+        (CredentialProvider::NoCredentials, _) => loader.no_credentials(),
+        (CredentialProvider::AwsProfile, Some(profile)) => loader.profile_name(profile),
+        _ => {
+            return Err(ParseError(
+                "profile must be specified if using aws-profile credential provider".to_string(),
+            ))
+        }
+    };
+
+    let config = config::Builder::from(&loader.load().await).build();
+
+    Ok(Client::from_conf(config))
+}
+
+/// Create the default S3 client.
 pub async fn default_s3_client() -> Result<Client> {
-    let config = load_defaults(BehaviorVersion::latest()).await;
-    Ok(Client::new(&config))
+    create_s3_client(&CredentialProvider::DefaultEnvironment, None, None, None).await
 }
 
 #[cfg(test)]
diff --git a/cloud-checksum/src/io/sums/aws.rs b/cloud-checksum/src/io/sums/aws.rs
index 7a017ab..616844c 100644
--- a/cloud-checksum/src/io/sums/aws.rs
+++ b/cloud-checksum/src/io/sums/aws.rs
@@ -6,8 +6,8 @@ use crate::checksum::file::Checksum;
 use crate::checksum::file::SumsFile;
 use crate::checksum::standard::StandardCtx;
 use crate::checksum::Ctx;
-use crate::error::Error::{AwsError, ParseError};
-use crate::error::{Error, Result};
+use crate::error::Error::ParseError;
+use crate::error::{ApiError, Error, Result};
 use crate::io::sums::ObjectSums;
 use crate::io::Provider;
 use aws_sdk_s3::operation::get_object::GetObjectError;
@@ -20,7 +20,7 @@ use aws_sdk_s3::Client;
 use aws_smithy_types::byte_stream::ByteStream;
 use base64::prelude::BASE64_STANDARD;
 use base64::Engine;
-use std::collections::HashMap;
+use std::collections::{HashMap, HashSet};
 use std::str::FromStr;
 use std::sync::Arc;
 use tokio::io::AsyncRead;
@@ -31,6 +31,7 @@ pub struct S3Builder {
     client: Option<Arc<Client>>,
     bucket: Option<String>,
     key: Option<String>,
+    avoid_get_object_attributes: bool,
 }
 
 impl S3Builder {
@@ -52,7 +53,13 @@ impl S3Builder {
         self
     }
 
-    fn get_components(self) -> Result<(Arc<Client>, String, String)> {
+    /// Avoid `GetObjectAttributes` calls.
+    pub fn with_avoid_get_object_attributes(mut self, avoid_get_object_attributes: bool) -> Self {
+        self.avoid_get_object_attributes = avoid_get_object_attributes;
+        self
+    }
+
+    fn get_components(self) -> Result<(Arc<Client>, String, String, bool)> {
         let error_fn =
             || ParseError("client, bucket and key are required in `S3Builder`".to_string());
 
@@ -60,6 +67,7 @@ impl S3Builder {
             self.client.ok_or_else(error_fn)?,
             self.bucket.ok_or_else(error_fn)?,
             self.key.ok_or_else(error_fn)?,
+            self.avoid_get_object_attributes,
         ))
     }
 
@@ -69,9 +77,11 @@ impl S3Builder {
     }
 }
 
-impl From<(Arc<Client>, String, String)> for S3 {
-    fn from((client, bucket, key): (Arc<Client>, String, String)) -> Self {
-        Self::new(client, bucket, key)
+impl From<(Arc<Client>, String, String, bool)> for S3 {
+    fn from(
+        (client, bucket, key, avoid_get_object_attributes): (Arc<Client>, String, String, bool),
+    ) -> Self {
+        Self::new(client, bucket, key, avoid_get_object_attributes)
     }
 }
 
@@ -82,18 +92,27 @@ pub struct S3 {
     bucket: String,
     key: String,
     get_object_attributes: Option<GetObjectAttributesOutput>,
-    head_object: HashMap<u64, HeadObjectOutput>,
+    head_object: HashMap<Option<u64>, HeadObjectOutput>,
+    api_errors: HashSet<ApiError>,
+    avoid_get_object_attributes: bool,
 }
 
 impl S3 {
     /// Create a new S3 object.
-    pub fn new(client: Arc<Client>, bucket: String, key: String) -> S3 {
+    pub fn new(
+        client: Arc<Client>,
+        bucket: String,
+        key: String,
+        avoid_get_object_attributes: bool,
+    ) -> S3 {
         Self {
             client,
             bucket,
             key,
             get_object_attributes: None,
             head_object: HashMap::new(),
+            api_errors: HashSet::new(),
+            avoid_get_object_attributes,
         }
     }
 
@@ -121,9 +140,13 @@ impl S3 {
 
     /// Get the `GetObjectAttributes` output for the target file. This caches the result in
     /// memory so that subsequent calls do not repeat the query.
-    pub async fn get_object_attributes(&mut self) -> Result<&GetObjectAttributesOutput> {
+    pub async fn get_object_attributes(&mut self) -> Option<&GetObjectAttributesOutput> {
+        if self.avoid_get_object_attributes {
+            return None;
+        }
+
         if let Some(ref attributes) = self.get_object_attributes {
-            return Ok(attributes);
+            return Some(attributes);
         }
 
         let attributes = self
@@ -136,14 +159,20 @@ impl S3 {
             .object_attributes(ObjectAttributes::ObjectSize)
             .object_attributes(ObjectAttributes::ObjectParts)
             .send()
-            .await?;
+            .await;
 
-        Ok(self.get_object_attributes.insert(attributes))
+        match attributes {
+            Ok(attributes) => Some(self.get_object_attributes.insert(attributes)),
+            Err(ref err) => {
+                self.api_errors.insert(ApiError::from(err));
+                None
+            }
+        }
     }
 
     /// Get the `HeadObjectOutput` output for the target file for a specific part. This caches
     /// the result in memory so that subsequent calls do not repeat the query for the same part.
-    pub async fn head_object(&mut self, part_number: u64) -> Result<&HeadObjectOutput> {
+    pub async fn head_object(&mut self, part_number: Option<u64>) -> Result<&HeadObjectOutput> {
         if self.head_object.contains_key(&part_number) {
             return Ok(&self.head_object[&part_number]);
         }
@@ -153,7 +182,7 @@ impl S3 {
             .head_object()
             .bucket(&self.bucket)
             .key(SumsFile::format_target_file(&self.key))
-            .part_number(i32::try_from(part_number)?)
+            .set_part_number(part_number.map(i32::try_from).transpose()?)
             .checksum_mode(ChecksumMode::Enabled)
             .send()
             .await?;
@@ -170,7 +199,11 @@ impl S3 {
     /// checksums (not including the `ETag`) are base64 encoded when returned from the SDK.
     /// The `ETag` is hex encoded.
     fn decode_sum(ctx: &StandardCtx, sum: String) -> Result<Vec<u8>> {
-        let sum = sum.split("-").next().unwrap_or_else(|| &sum);
+        let sum = sum
+            .trim_matches('"')
+            .split("-")
+            .next()
+            .unwrap_or_else(|| &sum);
 
         if Self::is_additional_checksum(ctx) {
             let data = BASE64_STANDARD
@@ -184,20 +217,19 @@ impl S3 {
         }
     }
 
-    /// Get the AWS checksum value from `GetObjectAttributes`.
+    /// Get the AWS checksum value from `HeadObject`.
     pub async fn aws_sums_from_ctx(&mut self, ctx: &StandardCtx) -> Result<Option<String>> {
-        let attributes = self.get_object_attributes().await?;
+        let head = self.head_object(None).await?;
+
         let sum = match ctx {
             // There are no part checksums for e_tags.
-            StandardCtx::MD5(_) => attributes.e_tag(),
+            StandardCtx::MD5(_) => head.e_tag(),
             // Every other checksum has part checksums available if uploaded using multipart uploads.
-            StandardCtx::SHA1(_) => attributes.checksum().and_then(|c| c.checksum_sha1()),
-            StandardCtx::SHA256(_) => attributes.checksum().and_then(|c| c.checksum_sha256()),
-            StandardCtx::CRC32(_, _) => attributes.checksum().and_then(|c| c.checksum_crc32()),
-            StandardCtx::CRC32C(_, _) => attributes.checksum().and_then(|c| c.checksum_crc32_c()),
-            StandardCtx::CRC64NVME(_, _) => {
-                attributes.checksum().and_then(|c| c.checksum_crc64_nvme())
-            }
+            StandardCtx::SHA1(_) => head.checksum_sha1(),
+            StandardCtx::SHA256(_) => head.checksum_sha256(),
+            StandardCtx::CRC32(_, _) => head.checksum_crc32(),
+            StandardCtx::CRC32C(_, _) => head.checksum_crc32_c(),
+            StandardCtx::CRC64NVME(_, _) => head.checksum_crc64_nvme(),
             _ => None,
         };
 
@@ -222,9 +254,11 @@ impl S3 {
 
     /// Get the AWS checksum parts from `GetObjectAttributes` parts output.
     pub async fn aws_parts_from_attributes(&mut self) -> Result<Option<Vec<Option<u64>>>> {
-        let parts = self
-            .get_object_attributes()
-            .await?
+        let Some(parts) = self.get_object_attributes().await else {
+            return Ok(None);
+        };
+
+        let parts = parts
             .object_parts()
             .map(|parts| {
                 let parts = parts
@@ -257,7 +291,7 @@ impl S3 {
         let mut part_sums = vec![];
 
         for part_number in 1..=total_parts {
-            let head_object = self.head_object(part_number).await?;
+            let head_object = self.head_object(Some(part_number)).await?;
 
             // The content length represents the part size. Return early if any of the content
             // lengths are not present, to avoid having empty part checksums.
@@ -282,21 +316,19 @@ impl S3 {
             return Ok(());
         };
 
-        // Get the file size, total part count and checksum type from the attributes. This is in
-        // a separate block to avoid mutably borrowing more than once later.
-        let (file_size, total_parts, checksum_type) = {
-            let attributes = self.get_object_attributes().await?;
-
-            let file_size = attributes.object_size().map(u64::try_from).transpose()?;
-            let (total_parts, checksum_type) = Self::parse_parts_and_type(sum.as_str())?;
-
-            (file_size, total_parts, checksum_type)
-        };
+        // Get the file size, total part count and checksum type from the head.
+        let file_size = self
+            .head_object(None)
+            .await?
+            .content_length()
+            .map(u64::try_from)
+            .transpose()?;
+        let (total_parts, checksum_type) = Self::parse_parts_and_type(sum.as_str())?;
 
         // Determine the parts if they exist.
         let parts = self.aws_parts_from_attributes().await?;
         // If there are no parts, try and find them using the total part count and head object.
-        // This should only trigger on `ETag`s.
+        // This should only trigger on `ETag`s or if `GetObjectAttributes` returns an error.
         let parts = match (parts, total_parts) {
             (Some(parts), _) => Some(parts),
             (None, Some(total_parts)) => self.aws_parts_from_head(total_parts).await?,
@@ -350,8 +382,8 @@ impl S3 {
     /// This is used to add as much information to the output sums file as possible.
     pub async fn sums_from_metadata(&mut self) -> Result<SumsFile> {
         // The target file metadata.
-        let attributes = self.get_object_attributes().await?;
-        let file_size = attributes.object_size().map(u64::try_from).transpose()?;
+        let head = self.head_object(None).await?;
+        let file_size = head.content_length().map(u64::try_from).transpose()?;
         let mut sums_file = SumsFile::default().with_size(file_size);
 
         // Add the individual checksums for each type.
@@ -369,7 +401,7 @@ impl S3 {
             .await?;
 
         if sums_file.checksums.is_empty() {
-            return Err(AwsError(
+            return Err(Error::aws_error(
                 "failed to create sums file from metadata".to_string(),
             ));
         }
@@ -379,7 +411,7 @@ impl S3 {
 
     /// Parse the number of parts and the checksum type from a string.
     pub fn parse_parts_and_type(s: &str) -> Result<(Option<u64>, ChecksumType)> {
-        let split = s.rsplit_once("-");
+        let split = s.trim_matches('\"').rsplit_once("-");
         if let Some((_, parts)) = split {
             let parts = u64::from_str(parts).map_err(|err| {
                 ParseError(format!("failed to parse parts from checksum: {}", err))
@@ -412,9 +444,9 @@ impl S3 {
     /// Get the object file size.
     async fn size(&mut self) -> Result<Option<u64>> {
         Ok(self
-            .get_object_attributes()
+            .head_object(None)
             .await?
-            .object_size
+            .content_length()
             .map(|size| size.try_into())
             .transpose()?)
     }
@@ -460,6 +492,10 @@ impl ObjectSums for S3 {
     fn location(&self) -> String {
         Provider::format_s3(&self.bucket, &self.key)
     }
+
+    fn api_errors(&self) -> HashSet<ApiError> {
+        self.api_errors.clone()
+    }
 }
 
 #[cfg(test)]
@@ -468,6 +504,7 @@ pub(crate) mod test {
     use crate::checksum::standard::test::EXPECTED_MD5_SUM;
     use crate::task::generate::test::generate_for;
     use crate::test::{TEST_FILE_NAME, TEST_FILE_SIZE};
+    use aws_sdk_s3::operation::head_object::builders::HeadObjectOutputBuilder;
     use aws_sdk_s3::types;
     use aws_sdk_s3::types::GetObjectAttributesParts;
     use aws_smithy_mocks_experimental::{mock, mock_client, Rule, RuleMode};
@@ -680,7 +717,18 @@ pub(crate) mod test {
 
         // If an additional checksum is present, then there is no need to call head object as the
         // parts are always in the get object attributes response.
-        mock_client!(aws_sdk_s3, RuleMode::Sequential, &[&get_object_attributes,])
+        mock_client!(
+            aws_sdk_s3,
+            RuleMode::Sequential,
+            &[
+                &head_object_size_rule(
+                    format!("\"{}\"", EXPECTED_MD5_SUM_4),
+                    Some(4),
+                    Some(EXPECTED_SHA256_SUM_4.to_string())
+                ),
+                &get_object_attributes,
+            ]
+        )
     }
 
     fn mock_multi_part_with_sha256() -> Client {
@@ -741,7 +789,35 @@ pub(crate) mod test {
 
         // If an additional checksum is present, then there is no need to call head object as the
         // parts are always in the get object attributes response.
-        mock_client!(aws_sdk_s3, RuleMode::Sequential, &[&get_object_attributes,])
+        mock_client!(
+            aws_sdk_s3,
+            RuleMode::Sequential,
+            &[
+                &head_object_size_rule(
+                    format!("\"{}\"", EXPECTED_MD5_SUM_5),
+                    Some(5),
+                    Some(EXPECTED_SHA256_SUM_5.to_string())
+                ),
+                &get_object_attributes,
+            ]
+        )
+    }
+
+    fn head_object_size_rule(
+        e_tag: String,
+        parts_count: Option<i32>,
+        sha256: Option<String>,
+    ) -> Rule {
+        mock!(Client::head_object)
+            .match_requests(|req| req.bucket() == Some("bucket") && req.key() == Some("key"))
+            .then_output(move || {
+                HeadObjectOutputBuilder::default()
+                    .e_tag(&e_tag)
+                    .set_parts_count(parts_count)
+                    .set_checksum_sha256(sha256.clone())
+                    .content_length(TEST_FILE_SIZE as i64)
+                    .build()
+            })
     }
 
     fn mock_multi_part_etag_only_different_part_sizes() -> Client {
@@ -763,6 +839,7 @@ pub(crate) mod test {
             aws_sdk_s3,
             RuleMode::Sequential,
             &[
+                &head_object_size_rule(format!("\"{}\"", EXPECTED_MD5_SUM_4), Some(4), None),
                 &get_object_attributes,
                 &head_object_rule(214748365),
                 &head_object_rule(214748365),
@@ -798,6 +875,7 @@ pub(crate) mod test {
             });
 
         vec![
+            head_object_size_rule(format!("\"{}\"", EXPECTED_MD5_SUM_5), Some(5), None),
             get_object_attributes,
             head_object_rule(214748365),
             head_object_rule(214748365),
@@ -822,23 +900,43 @@ pub(crate) mod test {
                     .build()
             });
 
-        mock_client!(aws_sdk_s3, RuleMode::Sequential, &[&get_object_attributes])
+        mock_client!(
+            aws_sdk_s3,
+            RuleMode::Sequential,
+            &[
+                &head_object_size_rule(
+                    format!("\"{}\"", EXPECTED_MD5_SUM),
+                    None,
+                    Some(EXPECTED_SHA256_SUM.to_string())
+                ),
+                &get_object_attributes
+            ]
+        )
     }
 
     fn mock_single_part_etag_only() -> Client {
         let get_object_attributes = mock_single_part_etag_only_rule();
 
-        mock_client!(aws_sdk_s3, RuleMode::Sequential, &[&get_object_attributes])
+        mock_client!(
+            aws_sdk_s3,
+            RuleMode::Sequential,
+            get_object_attributes.as_slice()
+        )
     }
 
-    pub(crate) fn mock_single_part_etag_only_rule() -> Rule {
-        mock!(Client::get_object_attributes)
-            .match_requests(move |req| req.bucket() == Some("bucket") && req.key() == Some("key"))
-            .then_output(|| {
-                GetObjectAttributesOutput::builder()
-                    .e_tag(EXPECTED_MD5_SUM)
-                    .object_size(TEST_FILE_SIZE as i64)
-                    .build()
-            })
+    pub(crate) fn mock_single_part_etag_only_rule() -> Vec<Rule> {
+        vec![
+            head_object_size_rule(format!("\"{}\"", EXPECTED_MD5_SUM), None, None),
+            mock!(Client::get_object_attributes)
+                .match_requests(move |req| {
+                    req.bucket() == Some("bucket") && req.key() == Some("key")
+                })
+                .then_output(|| {
+                    GetObjectAttributesOutput::builder()
+                        .e_tag(EXPECTED_MD5_SUM)
+                        .object_size(TEST_FILE_SIZE as i64)
+                        .build()
+                }),
+        ]
     }
 }
diff --git a/cloud-checksum/src/io/sums/file.rs b/cloud-checksum/src/io/sums/file.rs
index 23eca30..949aea6 100644
--- a/cloud-checksum/src/io/sums/file.rs
+++ b/cloud-checksum/src/io/sums/file.rs
@@ -3,8 +3,9 @@
 
 use crate::checksum::file::SumsFile;
 use crate::error::Error::ParseError;
-use crate::error::Result;
+use crate::error::{ApiError, Result};
 use crate::io::sums::ObjectSums;
+use std::collections::HashSet;
 use std::path::PathBuf;
 use tokio::fs;
 use tokio::io::{AsyncRead, AsyncReadExt};
@@ -92,7 +93,7 @@ impl File {
 #[async_trait::async_trait]
 impl ObjectSums for File {
     async fn sums_file(&mut self) -> Result<Option<SumsFile>> {
-        self.get_existing_sums().await
+        Ok(self.get_existing_sums().await?)
     }
 
     async fn reader(&mut self) -> Result<Box<dyn AsyncRead + Unpin + Send>> {
@@ -110,4 +111,8 @@ impl ObjectSums for File {
     fn location(&self) -> String {
         self.file.to_string()
     }
+
+    fn api_errors(&self) -> HashSet<ApiError> {
+        HashSet::new()
+    }
 }
diff --git a/cloud-checksum/src/io/sums/mod.rs b/cloud-checksum/src/io/sums/mod.rs
index c8fdd1c..15a4d5f 100644
--- a/cloud-checksum/src/io/sums/mod.rs
+++ b/cloud-checksum/src/io/sums/mod.rs
@@ -2,13 +2,14 @@
 //!
 
 use crate::checksum::file::SumsFile;
-use crate::error::Result;
+use crate::error::{ApiError, Result};
 use crate::io::sums::aws::S3Builder;
 use crate::io::sums::file::FileBuilder;
 use crate::io::{default_s3_client, Provider};
 use aws_sdk_s3::Client;
 use dyn_clone::DynClone;
 use futures_util::Stream;
+use std::collections::HashSet;
 use std::pin::Pin;
 use std::sync::Arc;
 use tokio::io::AsyncRead;
@@ -50,6 +51,9 @@ pub trait ObjectSums: DynClone {
 
     /// Get the location of the object.
     fn location(&self) -> String;
+
+    /// Any accumulated recoverable api errors.
+    fn api_errors(&self) -> HashSet<ApiError>;
 }
 
 dyn_clone::clone_trait_object!(ObjectSums);
@@ -58,6 +62,7 @@ dyn_clone::clone_trait_object!(ObjectSums);
 #[derive(Debug, Default)]
 pub struct ObjectSumsBuilder {
     client: Option<Arc<Client>>,
+    avoid_get_object_attributes: bool,
 }
 
 impl ObjectSumsBuilder {
@@ -76,6 +81,7 @@ impl ObjectSumsBuilder {
                         .with_key(key)
                         .with_bucket(bucket)
                         .with_client(client)
+                        .with_avoid_get_object_attributes(self.avoid_get_object_attributes)
                         .build()?,
                 ))
             }
@@ -87,4 +93,10 @@ impl ObjectSumsBuilder {
         self.client = client;
         self
     }
+
+    /// Avoid `GetObjectAttributes` calls.
+    pub fn with_avoid_get_object_attributes(mut self, avoid_get_object_attributes: bool) -> Self {
+        self.avoid_get_object_attributes = avoid_get_object_attributes;
+        self
+    }
 }
diff --git a/cloud-checksum/src/stats.rs b/cloud-checksum/src/stats.rs
index 92b8506..4523054 100644
--- a/cloud-checksum/src/stats.rs
+++ b/cloud-checksum/src/stats.rs
@@ -4,11 +4,12 @@
 use crate::checksum::file::Checksum;
 use crate::checksum::Ctx;
 use crate::cli::CopyMode;
+use crate::error::ApiError;
 use crate::task::check::{CheckTask, GroupBy};
 use crate::task::copy::CopyTask;
 use crate::task::generate::GenerateTask;
 use serde::{Deserialize, Serialize};
-use std::collections::BTreeMap;
+use std::collections::{BTreeMap, HashSet};
 use std::time::Duration;
 
 /// Stats from running a `generate` command.
@@ -22,6 +23,9 @@ pub struct GenerateStats {
     /// Stats from running `check` for comparability when computing sums with `--missing`.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) check_stats: Option<Box<CheckStats>>,
+    /// The API errors if there was permission issues for object attributes.
+    #[serde(skip_serializing_if = "HashSet::is_empty")]
+    pub(crate) api_errors: HashSet<ApiError>,
 }
 
 impl GenerateStats {
@@ -30,6 +34,7 @@ impl GenerateStats {
         elapsed_seconds: f64,
         stats: Vec<GenerateFileStats>,
         check_stats: Option<CheckStats>,
+        api_errors: HashSet<ApiError>,
     ) -> Self {
         Self {
             elapsed_seconds,
@@ -38,6 +43,7 @@ impl GenerateStats {
                 .filter(|stat| !stat.checksums_generated.0.is_empty())
                 .collect(),
             check_stats: check_stats.map(Box::new),
+            api_errors,
         }
     }
 }
@@ -133,6 +139,9 @@ pub struct CheckStats {
     /// Any generate stats computed if using `--missing`.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) generate_stats: Option<GenerateStats>,
+    /// The API errors if there was permission issues for object attributes.
+    #[serde(skip_serializing_if = "HashSet::is_empty")]
+    pub(crate) api_errors: HashSet<ApiError>,
 }
 
 impl CheckStats {
@@ -144,6 +153,7 @@ impl CheckStats {
         groups: Vec<Vec<String>>,
         updated: Vec<String>,
         generate_stats: Option<GenerateStats>,
+        api_errors: HashSet<ApiError>,
     ) -> Self {
         Self {
             elapsed_seconds,
@@ -152,6 +162,7 @@ impl CheckStats {
             groups,
             updated,
             generate_stats,
+            api_errors,
         }
     }
 
@@ -162,7 +173,7 @@ impl CheckStats {
         elapsed: Duration,
         generate_stats: Option<GenerateStats>,
     ) -> Self {
-        let (objects, compared, updated) = task.into_inner();
+        let (objects, compared, updated, api_errors) = task.into_inner();
 
         Self::new(
             elapsed.as_secs_f64(),
@@ -171,26 +182,11 @@ impl CheckStats {
             objects.to_groups(),
             updated,
             generate_stats,
+            api_errors,
         )
     }
 }
 
-/// An API error that could be returned from storage.
-#[derive(Serialize, Deserialize, Debug, Clone)]
-pub struct ApiError {
-    /// The error kind, e.g. `AccessDenied`.
-    pub(crate) error: String,
-    /// The error message.
-    pub(crate) message: String,
-}
-
-impl ApiError {
-    /// Create a new error.
-    pub fn new(error: String, message: String) -> Self {
-        Self { error, message }
-    }
-}
-
 /// Represents stats from a `copy` operation.
 #[derive(Serialize, Deserialize, Debug)]
 pub struct CopyStats {
@@ -219,8 +215,8 @@ pub struct CopyStats {
     /// The number of retries if there was permission issues for copying metadata or tags.
     pub(crate) n_retries: u64,
     /// The API errors if there was permission issues for copying metadata or tags.
-    #[serde(skip_serializing_if = "Vec::is_empty")]
-    pub(crate) api_errors: Vec<ApiError>,
+    #[serde(skip_serializing_if = "HashSet::is_empty")]
+    pub(crate) api_errors: HashSet<ApiError>,
     /// Stats from checking sums to ensure that the copy was successful.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) check_stats: Option<CheckStats>,
@@ -245,7 +241,7 @@ impl CopyStats {
             copy_mode: copy_task.copy_mode(),
             reason: check_stats.as_ref().and_then(Option::<ChecksumPair>::from),
             n_retries: copy_task.n_retries(),
-            api_errors: copy_task.api_errors().to_vec(),
+            api_errors: copy_task.api_errors(),
             check_stats,
         }
     }
diff --git a/cloud-checksum/src/task/check.rs b/cloud-checksum/src/task/check.rs
index ba133a2..3c2e42e 100644
--- a/cloud-checksum/src/task/check.rs
+++ b/cloud-checksum/src/task/check.rs
@@ -3,7 +3,7 @@
 
 use crate::checksum::file::{Checksum, SumsFile};
 use crate::checksum::Ctx;
-use crate::error::{Error, Result};
+use crate::error::{ApiError, Error, Result};
 use crate::io::sums::{ObjectSums, ObjectSumsBuilder};
 use crate::stats::{CheckComparison, ChecksumPair};
 use aws_sdk_s3::Client;
@@ -11,20 +11,35 @@ use clap::ValueEnum;
 use futures_util::future::join_all;
 use serde::{Deserialize, Serialize};
 use std::cmp::Ordering;
-use std::collections::{BTreeMap, BTreeSet};
+use std::collections::{BTreeMap, BTreeSet, HashSet};
 use std::fmt;
 use std::fmt::{Debug, Formatter};
 use std::hash::{DefaultHasher, Hash, Hasher};
 use std::sync::Arc;
 
 /// Build a check task.
-#[derive(Debug, Default)]
+#[derive(Debug)]
 pub struct CheckTaskBuilder {
     files: Vec<String>,
     sums_files: Vec<(String, SumsFile)>,
     group_by: GroupBy,
     update: bool,
-    client: Option<Arc<Client>>,
+    clients: Vec<Option<Arc<Client>>>,
+    avoid_get_object_attributes: bool,
+}
+
+impl Default for CheckTaskBuilder {
+    fn default() -> Self {
+        Self {
+            files: Default::default(),
+            sums_files: Default::default(),
+            group_by: Default::default(),
+            update: Default::default(),
+            // Ensure at least one element in the vector to repeat.
+            clients: vec![None],
+            avoid_get_object_attributes: Default::default(),
+        }
+    }
 }
 
 impl CheckTaskBuilder {
@@ -40,6 +55,12 @@ impl CheckTaskBuilder {
         self
     }
 
+    /// Set the S3 client to use for each input file.
+    pub fn with_clients(mut self, clients: Vec<Arc<Client>>) -> Self {
+        self.clients = clients.into_iter().map(Some).collect();
+        self
+    }
+
     /// Set the group by mode.
     pub fn with_group_by(mut self, group_by: GroupBy) -> Self {
         self.group_by = group_by;
@@ -54,38 +75,67 @@ impl CheckTaskBuilder {
 
     /// Set the S3 client to use.
     pub fn with_client(mut self, client: Arc<Client>) -> Self {
-        self.client = Some(client);
+        self.clients = vec![Some(client)];
+        self
+    }
+
+    /// Avoid `GetObjectAttributes` calls.
+    pub fn with_avoid_get_object_attributes(mut self, avoid_get_object_attributes: bool) -> Self {
+        self.avoid_get_object_attributes = avoid_get_object_attributes;
         self
     }
 
     /// Build a check task.
-    pub async fn build(self) -> Result<CheckTask> {
+    pub async fn build(mut self) -> Result<CheckTask> {
         let group_by = self.group_by;
 
-        let mut objects = join_all(self.files.into_iter().map(|file| {
-            let client = self.client.clone();
-
-            async move {
-                let mut sums = ObjectSumsBuilder::default()
-                    .set_client(client)
-                    .build(file.to_string())
-                    .await?;
-
-                let file_size = sums.file_size().await?;
-                let existing = sums
-                    .sums_file()
-                    .await?
-                    .unwrap_or_else(|| SumsFile::new(file_size, Default::default()));
-
-                Ok((
-                    SumsKey((existing, sums.location())),
-                    BTreeSet::from_iter(vec![State::ObjectSums(sums)]),
-                ))
-            }
-        }))
+        // Remove elements that are already set by in-memory sums files.
+        let in_memory = self
+            .sums_files
+            .iter()
+            .map(|(sums, _)| sums)
+            .collect::<Vec<_>>();
+        self.files.retain(|file| !in_memory.contains(&file));
+
+        let (objects, errors): (Vec<_>, Vec<_>) = join_all(
+            self.files
+                .into_iter()
+                .zip(self.clients.into_iter().cycle())
+                .map(|(file, client)| async move {
+                    let mut sums = ObjectSumsBuilder::default()
+                        .with_avoid_get_object_attributes(self.avoid_get_object_attributes)
+                        .set_client(client)
+                        .build(file.to_string())
+                        .await?;
+
+                    let file_size = sums.file_size().await?;
+                    let existing = sums
+                        .sums_file()
+                        .await?
+                        .unwrap_or_else(|| SumsFile::new(file_size, Default::default()));
+
+                    let errors = sums.api_errors();
+                    Ok((
+                        (
+                            SumsKey((existing, sums.location())),
+                            BTreeSet::from_iter(vec![State::ObjectSums(sums)]),
+                        ),
+                        errors,
+                    ))
+                }),
+        )
         .await
         .into_iter()
-        .collect::<Result<BTreeMap<_, _>>>()?;
+        .collect::<Result<Vec<_>>>()?
+        .into_iter()
+        .unzip();
+
+        let mut objects = BTreeMap::from_iter(objects);
+        let errors = HashSet::from_iter(
+            errors
+                .into_iter()
+                .flat_map(|err| err.into_iter().collect::<Vec<_>>()),
+        );
 
         for (location, sums) in self.sums_files {
             objects.insert(
@@ -98,6 +148,7 @@ impl CheckTaskBuilder {
             objects: CheckObjects(objects),
             group_by,
             update: self.update,
+            api_errors: errors,
             ..Default::default()
         })
     }
@@ -138,18 +189,28 @@ impl State {
         }
     }
 
+    /// Get the api errors.
+    pub fn api_errors(&mut self) -> HashSet<ApiError> {
+        match self {
+            State::ObjectSums(object) => object.api_errors(),
+            _ => HashSet::new(),
+        }
+    }
+
     /// Write the sums file to the location. If no object sums are used then this creates a new
     /// object sums to write the file.
     pub async fn write_sums_file(
         &self,
         sums: &SumsFile,
         client: Option<Arc<Client>>,
+        avoid_get_object_attributes: bool,
     ) -> Result<()> {
         match self {
             State::ObjectSums(object) => object.write_sums_file(sums).await,
             State::ExistingSums((location, _)) => {
                 ObjectSumsBuilder::default()
                     .set_client(client)
+                    .with_avoid_get_object_attributes(avoid_get_object_attributes)
                     .build(location.to_string())
                     .await?
                     .write_sums_file(sums)
@@ -256,6 +317,8 @@ pub struct CheckTask {
     compared_directly: Vec<CheckComparison>,
     updated: Vec<String>,
     client: Option<Arc<Client>>,
+    api_errors: HashSet<ApiError>,
+    avoid_get_object_attributes: bool,
 }
 
 impl CheckTask {
@@ -347,6 +410,7 @@ impl CheckTask {
     /// Runs the check task, returning the list of matching files.
     pub async fn run(self) -> Result<Self> {
         let update = self.update && matches!(self.group_by, GroupBy::Equality);
+        let avoid_get_object_attributes = self.avoid_get_object_attributes;
         let client = self.client.clone();
         let mut result = match self.group_by {
             GroupBy::Equality => Ok::<_, Error>(self.merge_same().await?),
@@ -360,8 +424,11 @@ impl CheckTask {
                     let mut location = location.clone();
                     let current = location.sums_file().await?;
 
+                    result.api_errors.extend(location.api_errors());
                     if current.as_ref() != Some(file) {
-                        location.write_sums_file(file, client.clone()).await?;
+                        location
+                            .write_sums_file(file, client.clone(), avoid_get_object_attributes)
+                            .await?;
                         updated_sums.push(location.location());
                     }
                 }
@@ -374,8 +441,20 @@ impl CheckTask {
     }
 
     /// Get the inner values.
-    pub fn into_inner(self) -> (CheckObjects, Vec<CheckComparison>, Vec<String>) {
-        (self.objects, self.compared_directly, self.updated)
+    pub fn into_inner(
+        self,
+    ) -> (
+        CheckObjects,
+        Vec<CheckComparison>,
+        Vec<String>,
+        HashSet<ApiError>,
+    ) {
+        (
+            self.objects,
+            self.compared_directly,
+            self.updated,
+            self.api_errors,
+        )
     }
 
     /// Get the inner state objects.
@@ -387,6 +466,19 @@ impl CheckTask {
     pub fn compared_directly(&self) -> &[CheckComparison] {
         self.compared_directly.as_slice()
     }
+
+    /// Get the api errors.
+    pub fn api_errors(self) -> HashSet<ApiError> {
+        self.api_errors.clone()
+    }
+
+    /// Does the state of the check task contain no checksums in any sums files.
+    pub fn is_empty(&self) -> bool {
+        self.objects.0.iter().all(|(key, _)| {
+            let SumsKey((sums, _)) = key;
+            sums.is_empty()
+        })
+    }
 }
 
 #[cfg(test)]
diff --git a/cloud-checksum/src/task/copy.rs b/cloud-checksum/src/task/copy.rs
index 650cbbc..c572d56 100644
--- a/cloud-checksum/src/task/copy.rs
+++ b/cloud-checksum/src/task/copy.rs
@@ -6,15 +6,15 @@ use crate::checksum::file::SumsFile;
 use crate::checksum::Ctx;
 use crate::cli::{CopyMode, MetadataCopy};
 use crate::error::Error::CopyError;
-use crate::error::{Error, Result};
+use crate::error::{ApiError, Error, Result};
 use crate::io::copy::{CopyResult, CopyState, MultiPartOptions, ObjectCopy, ObjectCopyBuilder};
 use crate::io::sums::ObjectSumsBuilder;
 use crate::io::Provider;
-use crate::stats::ApiError;
 use aws_sdk_s3::Client;
 use futures_util::future::join_all;
 use serde::{Deserialize, Serialize};
 use serde_json::to_string;
+use std::collections::HashSet;
 use std::future::Future;
 use std::sync::Arc;
 
@@ -30,8 +30,11 @@ pub struct CopyTaskBuilder {
     metadata_mode: MetadataCopy,
     tag_mode: MetadataCopy,
     copy_mode: CopyMode,
-    client: Option<Arc<Client>>,
+    source_client: Option<Arc<Client>>,
+    destination_client: Option<Arc<Client>>,
     concurrency: Option<usize>,
+    api_errors: HashSet<ApiError>,
+    avoid_get_object_attributes: bool,
 }
 
 /// Settings that determine the part size and additional checksums to use.
@@ -109,9 +112,15 @@ impl CopyTaskBuilder {
         self
     }
 
-    /// Set the S3 client to use for S3 copies.
-    pub fn with_client(mut self, client: Arc<Client>) -> Self {
-        self.client = Some(client);
+    /// Set the source S3 client to use for S3 copies.
+    pub fn with_source_client(mut self, client: Arc<Client>) -> Self {
+        self.source_client = Some(client);
+        self
+    }
+
+    /// Set the destination S3 client to use for S3 copies.
+    pub fn with_destination_client(mut self, client: Arc<Client>) -> Self {
+        self.destination_client = Some(client);
         self
     }
 
@@ -121,6 +130,12 @@ impl CopyTaskBuilder {
         self
     }
 
+    /// Avoid `GetObjectAttributes` calls.
+    pub fn with_avoid_get_object_attributes(mut self, avoid_get_object_attributes: bool) -> Self {
+        self.avoid_get_object_attributes = avoid_get_object_attributes;
+        self
+    }
+
     /// Return whether multipart is available.
     fn is_multipart(
         object_size: u64,
@@ -203,24 +218,27 @@ impl CopyTaskBuilder {
     /// 3. Use the `PART_SIZE_ORDERING` to find the best multipart copy part size if the size
     ///    reaches the `multipart_threshold` or otherwise use single part copies if possible.
     pub async fn use_settings(
-        self,
+        mut self,
         destination: Provider,
         destination_copy: &(dyn ObjectCopy + Send),
         state: &CopyState,
-    ) -> Result<CopySettings> {
+    ) -> Result<(Self, CopySettings)> {
         let size = state.size();
         let max_part_size = destination_copy.max_part_size();
         let max_parts = destination_copy.max_parts();
         let min_part_size = destination_copy.min_part_size();
 
-        // Only use the sums file if the size is not set.
+        // Only use the sums file if the size is not set at the source.
         let sums = if self.part_size.is_none() {
-            ObjectSumsBuilder::default()
-                .set_client(self.client.clone())
+            let mut object = ObjectSumsBuilder::default()
+                .with_avoid_get_object_attributes(self.avoid_get_object_attributes)
+                .set_client(self.source_client.clone())
                 .build(self.source.to_string())
-                .await?
-                .sums_file()
-                .await?
+                .await?;
+
+            self.api_errors.extend(object.api_errors());
+
+            object.sums_file().await?
         } else {
             None
         };
@@ -238,7 +256,7 @@ impl CopyTaskBuilder {
                 destination,
             )?;
             if self.part_size.is_none() {
-                return Ok(settings);
+                return Ok((self, settings));
             } else {
                 Some(settings)
             }
@@ -265,7 +283,10 @@ impl CopyTaskBuilder {
                     max_part_size,
                     min_part_size,
                 ) {
-                    Ok(CopySettings::new(Some(part_size), additional_ctx, size))
+                    Ok((
+                        self,
+                        CopySettings::new(Some(part_size), additional_ctx, size),
+                    ))
                 } else {
                     Err(CopyError(format!(
                         "invalid part size `{}` and threshold `{}` for the object size `{}`",
@@ -275,7 +296,7 @@ impl CopyTaskBuilder {
             }
         }
 
-        let err = || {
+        let err_fn = || {
             CopyError(format!(
                 "failed to find a valid part size for the threshold `{}` with object size `{}`",
                 threshold, size
@@ -288,20 +309,23 @@ impl CopyTaskBuilder {
             });
 
             return if let Some(part_size) = part_size {
-                Ok(CopySettings::new(Some(*part_size), additional_ctx, size))
+                Ok((
+                    self,
+                    CopySettings::new(Some(*part_size), additional_ctx, size),
+                ))
             } else {
-                Err(err())
+                Err(err_fn())
             };
         }
 
         // Otherwise use single part if possible.
         if Self::is_single_part(size, max_part_size) {
-            return Ok(CopySettings::new(None, additional_ctx, size));
+            return Ok((self, CopySettings::new(None, additional_ctx, size)));
         }
 
         // This condition may occur if the size is greater than the possible single part upload
         // limit but lower than the threshold.
-        Err(err())
+        Err(err_fn())
     }
 
     /// Build a generate task.
@@ -325,11 +349,11 @@ impl CopyTaskBuilder {
             CopyMode::DownloadUpload
         };
 
-        let (source_copy, destination_copy) = if is_same_provider {
+        let (source_copy, destination_copy) = if copy_mode.is_server_side() {
             let source = ObjectCopyBuilder::default()
                 .with_copy_metadata(self.metadata_mode)
                 .with_copy_tags(self.tag_mode)
-                .set_client(self.client.clone())
+                .set_client(self.source_client.clone())
                 .set_source(Some(source.clone()))
                 .set_destination(Some(destination.clone()))
                 .build()
@@ -341,25 +365,27 @@ impl CopyTaskBuilder {
                 ObjectCopyBuilder::default()
                     .with_copy_metadata(self.metadata_mode)
                     .with_copy_tags(self.tag_mode)
-                    .set_client(self.client.clone())
+                    .set_client(self.source_client.clone())
                     .set_source(Some(source.clone()))
                     .build()
                     .await?,
                 ObjectCopyBuilder::default()
                     .with_copy_metadata(self.metadata_mode)
                     .with_copy_tags(self.tag_mode)
-                    .set_client(self.client.clone())
+                    .set_client(self.destination_client.clone())
                     .set_destination(Some(destination.clone()))
                     .build()
                     .await?,
             )
         };
+
         let state = source_copy.initialize_state().await?;
 
         let concurrency = self
             .concurrency
             .ok_or_else(|| CopyError("concurrency not set".to_string()))?;
-        let settings = self
+
+        let (this, settings) = self
             .use_settings(destination.clone(), destination_copy.as_ref(), &state)
             .await?;
 
@@ -377,7 +403,7 @@ impl CopyTaskBuilder {
             destination,
             bytes_transferred: 0,
             n_retries: 0,
-            api_errors: vec![],
+            api_errors: this.api_errors,
         };
 
         Ok(copy_task)
@@ -412,7 +438,7 @@ pub struct CopyTask {
     ordered_upload: bool,
     bytes_transferred: u64,
     n_retries: u64,
-    api_errors: Vec<ApiError>,
+    api_errors: HashSet<ApiError>,
 }
 
 impl CopyTask {
@@ -598,7 +624,8 @@ impl CopyTask {
 
         self.bytes_transferred = bytes_transferred;
         self.n_retries = n_retries;
-        self.api_errors = api_errors;
+        self.api_errors
+            .extend::<HashSet<ApiError>>(HashSet::from_iter(api_errors));
 
         Ok(self)
     }
@@ -624,8 +651,8 @@ impl CopyTask {
     }
 
     /// Get the api errors.
-    pub fn api_errors(&self) -> &[ApiError] {
-        self.api_errors.as_slice()
+    pub fn api_errors(&self) -> HashSet<ApiError> {
+        self.api_errors.clone()
     }
 
     /// Get the number of retries.
@@ -685,7 +712,7 @@ pub(crate) mod test {
     async fn copy_settings() -> Result<()> {
         let test_file = TestFileBuilder::default().generate_test_defaults()?;
 
-        let single_part = vec![mock_single_part_etag_only_rule()];
+        let single_part = mock_single_part_etag_only_rule();
         let multipart = mock_multi_part_etag_only_rule();
 
         let builder = CopyTaskBuilder::default()
@@ -696,21 +723,21 @@ pub(crate) mod test {
         let lt_threshold = builder
             .clone()
             .with_multipart_threshold(Some(TEST_FILE_SIZE + 1))
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
         assert_eq!(lt_threshold.build().await?.part_size, None);
 
         // S3 to S3 will always prefer the original upload settings so even if the size is greater than the
         // threshold, it should still be single part.
         let gt_threshold = builder
             .clone()
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
         assert_eq!(gt_threshold.build().await?.part_size, None);
 
         // If it was originally multipart, it should prefer that even if below the threshold.
         let multipart_lt_threshold = builder
             .clone()
             .with_multipart_threshold(Some(TEST_FILE_SIZE + 1))
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, multipart.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, multipart.as_slice())));
         assert_eq!(
             multipart_lt_threshold.build().await?.part_size,
             Some(214748365)
@@ -718,7 +745,7 @@ pub(crate) mod test {
 
         let multipart_gt_threshold = builder
             .clone()
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, multipart.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, multipart.as_slice())));
         assert_eq!(
             multipart_gt_threshold.build().await?.part_size,
             Some(214748365)
@@ -728,12 +755,12 @@ pub(crate) mod test {
         let part_size_set = builder
             .clone()
             .with_part_size(Some(5242880))
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
         assert_eq!(part_size_set.build().await?.part_size, Some(5242880));
         let part_size_set_multipart = builder
             .clone()
             .with_part_size(Some(5242880))
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, multipart.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, multipart.as_slice())));
         assert_eq!(
             part_size_set_multipart.build().await?.part_size,
             Some(5242880)
@@ -762,13 +789,13 @@ pub(crate) mod test {
         let part_size_err_max = builder
             .clone()
             .with_part_size(Some(60000000000))
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
         assert!(part_size_err_max.build().await.is_err());
         // If the part size exceeds the limits, this should be an error.
         let part_size_err_min = builder
             .clone()
             .with_part_size(Some(1))
-            .with_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
+            .with_source_client(Arc::new(mock_size(TEST_FILE_SIZE, single_part.as_slice())));
         assert!(part_size_err_min.build().await.is_err());
 
         Ok(())
diff --git a/cloud-checksum/src/task/generate.rs b/cloud-checksum/src/task/generate.rs
index d49755b..fe5cef6 100644
--- a/cloud-checksum/src/task/generate.rs
+++ b/cloud-checksum/src/task/generate.rs
@@ -4,7 +4,7 @@
 use crate::checksum::file::{Checksum, SumsFile};
 use crate::checksum::Ctx;
 use crate::error::Error::GenerateError;
-use crate::error::{Error, Result};
+use crate::error::{ApiError, Error, Result};
 use crate::io::sums::channel::ChannelReader;
 use crate::io::sums::{ObjectSums, ObjectSumsBuilder, SharedReader};
 use crate::task::check::{CheckObjects, SumsKey};
@@ -33,6 +33,7 @@ pub struct GenerateTaskBuilder {
     capacity: usize,
     write: bool,
     client: Option<Arc<Client>>,
+    avoid_get_object_attributes: bool,
 }
 
 impl GenerateTaskBuilder {
@@ -73,8 +74,13 @@ impl GenerateTaskBuilder {
     }
 
     /// Set the S3 client to use.
-    pub fn with_client(mut self, client: Arc<Client>) -> Self {
-        self.client = Some(client);
+    pub fn with_client(self, client: Arc<Client>) -> Self {
+        self.set_client(Some(client))
+    }
+
+    /// Set the S3 client to use.
+    pub fn set_client(mut self, client: Option<Arc<Client>>) -> Self {
+        self.client = client;
         self
     }
 
@@ -89,10 +95,17 @@ impl GenerateTaskBuilder {
         self
     }
 
+    /// Avoid `GetObjectAttributes` calls.
+    pub fn with_avoid_get_object_attributes(mut self, avoid_get_object_attributes: bool) -> Self {
+        self.avoid_get_object_attributes = avoid_get_object_attributes;
+        self
+    }
+
     /// Build a generate task.
     pub async fn build(mut self) -> Result<GenerateTask> {
         let mut sums = ObjectSumsBuilder::default()
             .set_client(self.client)
+            .with_avoid_get_object_attributes(self.avoid_get_object_attributes)
             .build(self.input_file_name.to_string())
             .await?;
 
@@ -308,6 +321,11 @@ impl GenerateTask {
         )
     }
 
+    /// Get the api errors.
+    pub fn api_errors(&self) -> HashSet<ApiError> {
+        self.object_sums.api_errors()
+    }
+
     /// Return the computed sums file.
     pub fn sums_file(&self) -> &SumsFile {
         &self.output
@@ -423,7 +441,7 @@ pub(crate) mod test {
             .with_group_by(GroupBy::Comparability)
             .build()
             .await?;
-        let (objects, _, _) = check.run().await?.into_inner();
+        let (objects, _, _, _) = check.run().await?.into_inner();
 
         let result = SumCtxPairs::from_comparable(objects)?.unwrap();
 
diff --git a/cloud-checksum/tests/copy.rs b/cloud-checksum/tests/copy.rs
index 518f877..a3cd8dd 100644
--- a/cloud-checksum/tests/copy.rs
+++ b/cloud-checksum/tests/copy.rs
@@ -6,9 +6,10 @@ use anyhow::Result;
 use aws_sdk_s3::operation::head_object::HeadObjectOutput;
 use aws_sdk_s3::types::ChecksumMode;
 use aws_sdk_s3::Client;
-use cloud_checksum::cli::Command;
-use cloud_checksum::io::{default_s3_client, Provider};
+use cloud_checksum::cli::{Command, CredentialProvider};
+use cloud_checksum::io::{create_s3_client, Provider};
 use cloud_checksum::test::TestFileBuilder;
+use dotenvy::dotenv;
 use envy::prefixed;
 use serde::Deserialize;
 use std::fs::File;
@@ -20,10 +21,13 @@ use tempfile::TempDir;
 #[derive(Debug, Deserialize)]
 struct TestConfig {
     bucket_uri: String,
+    endpoint_url: Option<String>,
 }
 
 impl TestConfig {
     fn load() -> Result<Self> {
+        dotenv()?;
+
         let mut env: Self = prefixed("CLOUD_CHECKSUM_TEST_").from_env()?;
 
         env.bucket_uri = env
@@ -32,12 +36,29 @@ impl TestConfig {
             .unwrap_or(&env.bucket_uri)
             .to_string();
 
+        env.endpoint_url = env
+            .endpoint_url
+            .map(|url| url.strip_suffix("/").unwrap_or(&url).to_string());
+
         Ok(env)
     }
 
     fn format_uri(&self, path: &str) -> String {
         format!("{}/{}", self.bucket_uri, path)
     }
+
+    fn set_endpoint_url(&self, mut commands: Vec<String>) -> Vec<String> {
+        if let Some(endpoint_url) = &self.endpoint_url {
+            commands.extend([
+                "--source-endpoint-url".to_string(),
+                endpoint_url.to_string(),
+                "--destination-endpoint-url".to_string(),
+                endpoint_url.to_string(),
+            ]);
+        }
+
+        commands
+    }
 }
 
 #[ignore]
@@ -45,7 +66,13 @@ impl TestConfig {
 async fn copy_test() -> Result<()> {
     let config = TestConfig::load()?;
     let file = TestFileBuilder::default().generate_bench_defaults()?;
-    let client = default_s3_client().await?;
+    let client = create_s3_client(
+        &CredentialProvider::DefaultEnvironment,
+        None,
+        None,
+        config.endpoint_url.as_deref(),
+    )
+    .await?;
 
     // Local to S3.
     local_s3_multipart(file.as_path(), &config, &client).await?;
@@ -67,7 +94,7 @@ async fn local_s3_multipart(file: &Path, config: &TestConfig, client: &Client) -
     let uri = config.format_uri("multipart");
     let file = file.to_string_lossy();
 
-    execute_multipart(file.as_ref(), uri.as_ref()).await;
+    execute_multipart(file.as_ref(), uri.as_ref(), config).await;
 
     let head = get_head_object(client, uri.as_ref()).await?;
     assert_head_multipart(head);
@@ -80,7 +107,7 @@ async fn local_s3_single_part(file: &Path, config: &TestConfig, client: &Client)
     let uri = config.format_uri("single_part");
     let file = file.to_string_lossy();
 
-    execute_single_part(file.as_ref(), uri.as_ref()).await;
+    execute_single_part(file.as_ref(), uri.as_ref(), config).await;
 
     let head = get_head_object(client, uri.as_ref()).await?;
     assert_head_single_part(head);
@@ -93,7 +120,7 @@ async fn s3_s3_multipart(config: &TestConfig, client: &Client) -> Result<()> {
     let uri = config.format_uri("multipart");
     let destination = config.format_uri("multipart_copy");
 
-    execute_multipart(uri.as_ref(), destination.as_ref()).await;
+    execute_multipart(uri.as_ref(), destination.as_ref(), config).await;
 
     let head = get_head_object(client, destination.as_ref()).await?;
     assert_head_multipart(head);
@@ -106,7 +133,7 @@ async fn s3_s3_single_part(config: &TestConfig, client: &Client) -> Result<()> {
     let uri = config.format_uri("single_part");
     let destination = config.format_uri("single_part_copy");
 
-    execute_single_part(uri.as_ref(), destination.as_ref()).await;
+    execute_single_part(uri.as_ref(), destination.as_ref(), config).await;
 
     let head = get_head_object(client, destination.as_ref()).await?;
     assert_head_single_part(head);
@@ -120,7 +147,7 @@ async fn s3_local_multipart(original: &Path, config: &TestConfig) -> Result<()>
     let tmp = TempDir::new()?;
     let copy_to = tmp.path().join("multipart_copy");
 
-    execute_multipart(uri.as_ref(), copy_to.to_string_lossy().as_ref()).await;
+    execute_multipart(uri.as_ref(), copy_to.to_string_lossy().as_ref(), config).await;
     assert_original(
         original.to_str().unwrap(),
         copy_to.to_string_lossy().as_ref(),
@@ -135,7 +162,7 @@ async fn s3_local_single_part(original: &Path, config: &TestConfig) -> Result<()
     let tmp = TempDir::new()?;
     let copy_to = tmp.path().join("single_part_copy");
 
-    execute_single_part(uri.as_ref(), copy_to.to_string_lossy().as_ref()).await;
+    execute_single_part(uri.as_ref(), copy_to.to_string_lossy().as_ref(), config).await;
     assert_original(
         original.to_str().unwrap(),
         copy_to.to_string_lossy().as_ref(),
@@ -167,8 +194,8 @@ async fn get_head_object(client: &Client, url: &str) -> Result<HeadObjectOutput>
     Ok(head)
 }
 
-async fn execute_multipart(from: &str, to: &str) {
-    let commands = [
+async fn execute_multipart(from: &str, to: &str, config: &TestConfig) {
+    let mut commands = [
         "cloud-checksum",
         "copy",
         from,
@@ -177,12 +204,19 @@ async fn execute_multipart(from: &str, to: &str) {
         "5MiB",
         "--part-size",
         "5MiB",
-    ];
+        "--tag-mode",
+        "best-effort",
+    ]
+    .into_iter()
+    .map(|s| s.to_string())
+    .collect();
+    commands = config.set_endpoint_url(commands);
+
     execute_command(&commands).await;
 }
 
-async fn execute_single_part(from: &str, to: &str) {
-    let commands = [
+async fn execute_single_part(from: &str, to: &str, config: &TestConfig) {
+    let mut commands = [
         "cloud-checksum",
         "copy",
         from,
@@ -191,7 +225,14 @@ async fn execute_single_part(from: &str, to: &str) {
         "20MiB",
         "--multipart-threshold",
         "20MiB",
-    ];
+        "--tag-mode",
+        "best-effort",
+    ]
+    .into_iter()
+    .map(|s| s.to_string())
+    .collect();
+    commands = config.set_endpoint_url(commands);
+
     execute_command(&commands).await;
 }
 
@@ -200,7 +241,13 @@ fn assert_head_multipart(head: HeadObjectOutput) {
         head.e_tag,
         Some("\"ec1e29805585d04a93eb8cf464b68c43-2\"".to_string())
     );
-    assert_eq!(head.checksum_crc64_nvme, Some("yM/EwMxFxsE=".to_string()));
+
+    if let Some(checksum) = head.checksum_crc64_nvme {
+        assert_eq!(checksum, "yM/EwMxFxsE=".to_string());
+    }
+    if let Some(checksum) = head.checksum_crc32_c {
+        assert_eq!(checksum, "4VjD4A==".to_string());
+    }
 }
 
 fn assert_head_single_part(head: HeadObjectOutput) {
@@ -208,10 +255,16 @@ fn assert_head_single_part(head: HeadObjectOutput) {
         head.e_tag,
         Some("\"617808065bb1a8be2755f9be0c0ac769\"".to_string())
     );
-    assert_eq!(head.checksum_crc64_nvme, Some("yM/EwMxFxsE=".to_string()));
+
+    if let Some(checksum) = head.checksum_crc64_nvme {
+        assert_eq!(checksum, "yM/EwMxFxsE=".to_string());
+    }
+    if let Some(checksum) = head.checksum_crc32_c {
+        assert_eq!(checksum, "4VjD4A==".to_string());
+    }
 }
 
-async fn execute_command(commands: &[&str]) {
+async fn execute_command(commands: &[String]) {
     let args = Command::parse_from_iter(commands).unwrap();
     args.execute().await.unwrap();
 }


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This PR enhances AWS credentials options by introducing separate source and destination clients and refining error handling, API statistics and S3 interactions. Key changes include:
- Addition of new global credentials options including source/destination settings.
- Splitting of S3 client usage into separate source and destination clients for copy operations.
- Updates to error handling and API error collection by switching from vector to HashSet and refining AWS error processing.

### Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

<details>
<summary>Show a summary per file</summary>

| File                                  | Description                                                   |
| ------------------------------------- | ------------------------------------------------------------- |
| cloud-checksum/src/task/generate.rs   | Introduces new client setters and an api_errors accessor.     |
| cloud-checksum/src/task/copy.rs        | Splits S3 client usage to support distinct source and destination credentials. |
| cloud-checksum/src/task/check.rs       | Updates client handling to support multiple clients.          |
| cloud-checksum/src/stats.rs             | Refactors API errors type from Vec to HashSet.                |
| cloud-checksum/src/io/*               | Adjusts error and AWS API interactions to match new designs.    |
| cloud-checksum/src/cli.rs              | Adds global credentials options and updates client creation logic.|
| cloud-checksum/src/error.rs            | Refines AWS error handling with structured ApiError.         |
| cloud-checksum/src/checksum/aws_etag.rs  | Minor update to error creation in AWS ETag context.            |
</details>



<details>
<summary>Comments suppressed due to low confidence (1)</summary>

**cloud-checksum/src/task/generate.rs:75**
* [nitpick] There are two similar methods (with_client and set_client) that perform nearly the same function for setting the client. Consider clarifying their differences in the documentation or consolidating them to reduce confusion.
```
pub fn with_client(mut self, client: Arc<Client>) -> Self {
```
</details>


state: COMMENTED
author_association: NONE
submitted_at: 2025-05-09T04:51:43Z
user: login: andrewpatto
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-09T06:26:50Z
user: login: brainstorm
body: LGTM!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T00:06:15Z
user: login: andrewpatto
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-12T01:27:57Z
user: login: mmalenic
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T00:53:04Z

Review comments:
diff_hunk: @@ -672,3 +769,72 @@ pub struct Output {
     #[arg(global = true, long, env)]
     pub write_sums_file: bool,
 }
+
+/// Options related to credentials. Options prefixed with `source_` affect `check`, `generate` and
+/// the source of a `copy` command. These options also have an alias without the prefix as they are
+/// used in all commands. Options prefixed with `destination_` only affect the destination of a
+/// `copy` command.
+#[derive(Args, Debug)]
+#[group(required = false)]
+pub struct Credentials {
+    /// The credentials source credentials to use. This affects the credentials used for `check`
+    /// `generate` and the source of a `copy` operation.
+    #[arg(
+        global = true,
+        long,
+        env,
+        default_value = "default-environment",
+        alias = "credential-provider"
+    )]
+    pub source_credential_provider: CredentialProvider,
+    /// The destination credentials to use. This only affects the credentials used for the
+    /// destination of a `copy` operation.
+    #[arg(global = true, long, env, default_value = "default-environment")]
+    pub destination_credential_provider: CredentialProvider,
+    /// The source profile to use if the source credential provider is `aws-profile`.
+    /// This must be specified if using `aws-profile`.
+    #[arg(global = true, long, env, alias = "profile")]
+    pub source_profile: Option<String>,
+    /// The destinatiom profile to use if the destination credential provider is `aws-profile`.
path: cloud-checksum/src/cli.rs
user: login: Copilot
body: The word 'destinatiom' is misspelled. Please correct it to 'destination'.
```suggestion
    /// The destination profile to use if the destination credential provider is `aws-profile`.
```
created_at: 2025-05-09T04:51:42Z
author_association: NONE
diff_hunk: @@ -84,17 +134,97 @@ impl From<serde_json::Error> for Error {
     }
 }
 
-impl<T> From<SdkError<T>> for Error
+impl From<byte_stream::error::Error> for Error {
+    fn from(err: byte_stream::error::Error) -> Self {
+        Self::IOError(io::Error::other(err))
+    }
+}
+
+/// An API error that could be returned from storage.
+#[derive(Serialize, Deserialize, Debug, Clone, Eq, PartialOrd, PartialEq, Ord, Hash)]
+pub struct ApiError {
+    /// The error kind, e.g. `AccessDenied`.
+    pub(crate) code: String,
+    /// The API call.
+    pub(crate) call: String,
+    /// The error message.
+    pub(crate) message: String,
+}
+
+impl ApiError {
+    /// Create a new error.
+    pub fn new(code: String, call: String, message: String) -> Self {
+        Self {
+            code,
+            call,
+            message,
+        }
+    }
+
+    /// Check if the error is an access denied error.
+    pub fn is_access_denied(&self) -> bool {
+        self.code == "AccessDenied"
+    }
+}
+
+impl<T> From<(&SdkError<T, HttpResponse>, String)> for ApiError
 where
-    T: Send + Sync + error::Error + CreateUnhandledError + 'static,
+    T: ProvideErrorMetadata + CreateUnhandledError + error::Error + Send + Sync + 'static,
 {
-    fn from(err: SdkError<T>) -> Self {
-        Self::AwsError(err.into_service_error().to_string())
+    fn from((err, call): (&SdkError<T, HttpResponse>, String)) -> Self {
+        Self::new(
+            err.code().unwrap_or("Unknown").to_string(),
+            call,
+            err.message()
+                .map(|msg| msg.to_string())
+                .or_else(|| err.as_service_error().map(|err| err.to_string()))
+                .unwrap_or_else(|| DisplayErrorContext(&err).to_string()),
+        )
     }
 }
 
-impl From<byte_stream::error::Error> for Error {
-    fn from(err: byte_stream::error::Error) -> Self {
-        Self::AwsError(err.to_string())
+impl Display for ApiError {
+    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
+        write!(f, "{} for {}: {}", self.code, self.call, self.message)
     }
 }
+
path: cloud-checksum/src/error.rs
user: login: brainstorm
body: Interesting macro, @mmalenic, I'm surprised that this is necessary, I'd have expected that aws-sdk-rust would help more with error mgmt?
created_at: 2025-05-12T00:05:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -818,6 +849,11 @@ pub struct Credentials {
     /// that has an S3-compatible storage API.
     #[arg(global = true, long, env)]
     pub destination_endpoint_url: Option<String>,
+    /// Avoid `GetObjectAttributes` calls when determining sums. `HeadObject` will be used as a
+    /// fallback. `GetObjectAttributes` is preferred over `HeadObject` because it only requires
+    /// a single call rather than a call for each part.
+    #[arg(global = true, long, env)]
+    pub avoid_get_object_attributes: bool,
path: cloud-checksum/src/cli.rs
user: login: andrewpatto
body: definitely not for this PR - but I think I eventually would think of this as "endpoint features". So there might be a variety of kind of feature flags that can be switched on and off for endpoints.

e.g.
`--source-endpoint-features=-objectatrributes,+lifeycles`



created_at: 2025-05-12T01:27:57Z
author_association: MEMBER
diff_hunk: @@ -84,17 +134,97 @@ impl From<serde_json::Error> for Error {
     }
 }
 
-impl<T> From<SdkError<T>> for Error
+impl From<byte_stream::error::Error> for Error {
+    fn from(err: byte_stream::error::Error) -> Self {
+        Self::IOError(io::Error::other(err))
+    }
+}
+
+/// An API error that could be returned from storage.
+#[derive(Serialize, Deserialize, Debug, Clone, Eq, PartialOrd, PartialEq, Ord, Hash)]
+pub struct ApiError {
+    /// The error kind, e.g. `AccessDenied`.
+    pub(crate) code: String,
+    /// The API call.
+    pub(crate) call: String,
+    /// The error message.
+    pub(crate) message: String,
+}
+
+impl ApiError {
+    /// Create a new error.
+    pub fn new(code: String, call: String, message: String) -> Self {
+        Self {
+            code,
+            call,
+            message,
+        }
+    }
+
+    /// Check if the error is an access denied error.
+    pub fn is_access_denied(&self) -> bool {
+        self.code == "AccessDenied"
+    }
+}
+
+impl<T> From<(&SdkError<T, HttpResponse>, String)> for ApiError
 where
-    T: Send + Sync + error::Error + CreateUnhandledError + 'static,
+    T: ProvideErrorMetadata + CreateUnhandledError + error::Error + Send + Sync + 'static,
 {
-    fn from(err: SdkError<T>) -> Self {
-        Self::AwsError(err.into_service_error().to_string())
+    fn from((err, call): (&SdkError<T, HttpResponse>, String)) -> Self {
+        Self::new(
+            err.code().unwrap_or("Unknown").to_string(),
+            call,
+            err.message()
+                .map(|msg| msg.to_string())
+                .or_else(|| err.as_service_error().map(|err| err.to_string()))
+                .unwrap_or_else(|| DisplayErrorContext(&err).to_string()),
+        )
     }
 }
 
-impl From<byte_stream::error::Error> for Error {
-    fn from(err: byte_stream::error::Error) -> Self {
-        Self::AwsError(err.to_string())
+impl Display for ApiError {
+    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
+        write!(f, "{} for {}: {}", self.code, self.call, self.message)
     }
 }
+
path: cloud-checksum/src/error.rs
user: login: mmalenic
body: It does help but it's a bit all or nothing, you can either print something like "Aws Error: unhandled" which gives no info, or the full error context with `DisplayErrorContext` which gives too much unnecessary information.

The point of this macro isn't so much about improving the error message (lines 174 - 182 of this file mostly do that), it's more to `stringify` the API call that caused the error which wouldn't be possible with generics. E.g. when an error is found, this code will show that it came from a `HeadObject` call.

For some reason, AWS errors never print what API call caused the error, they only print the error code and things like request ids if using the full context. 
created_at: 2025-05-22T00:53:04Z
author_association: MEMBER

PR comments:
user: login: mmalenic
created_at: 2025-05-09T04:58:32Z
body: Also possibly closes https://github.com/umccr/cloud-checksum/issues/33. Might want to keep it open, the options could change a bit with google cloud support.
author_association: MEMBER
user: login: andrewpatto
created_at: 2025-05-09T06:24:32Z
body: > Some credential combinations and endpoints don't support calls like GetObjectAttributes. Now, a call to GetObjectAttributes shows as a recoverable error in the api_errors stats fields and HeadObject is used if GetObjectAttributes fails.

If I know my endpoint doesn't support GetObjectAttributes can I downgrade the "discovery" call via command line?

(just thinking for a case where we are copying 1 million files - that's a lot of recoverable error calls that could be avoided)

author_association: MEMBER
user: login: mmalenic
created_at: 2025-05-11T23:49:42Z
body: >If I know my endpoint doesn't support GetObjectAttributes can I downgrade the "discovery" call via command line?

Yeah that's not a bad idea too. I've added it as an option on the command line now.
author_association: MEMBER
user: login: mmalenic
created_at: 2025-05-12T06:18:35Z
body: I've also added `endpoint_url` to the integration tests, so that they can be used to test S3-compatible endpoints.
author_association: MEMBER