neulab/tldr huggingface dataset:
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / Alias for an existing remote \ "alias" [snip] Storage> alias Remote or path to alias. Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/local/path". remote> /mnt/storage/backup Remote config -------------------- [remote] remote = /mnt/storage/backup -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y Current remotes:
Name                 Type ====                 ==== remote               alias
e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q
List directories in top level in /mnt/storage/backup
List all the files in /mnt/storage/backup
Copy another local directory to the alias directory called source
rclone copy /home/source remote:source
Standard options Here are the standard options specific to alias (Alias for an existing remote).
--alias-remote Remote or path to alias.
Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/local/path".
• Config: remote
• Env Var: RCLONE_ALIAS_REMOTE
Amazon Drive Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage service run by Amazon for consumers.
Status Important: rclone supports Amazon Drive only if you have your own set of API keys.  Unfortunately the Amazon Drive developer program (https://developer.amazon.com/amazon-drive) is now closed to new entries so if you don't already have your own set of keys you will not be able to use rclone with Amazon Drive.
For the history on why rclone no longer has a set of Amazon Drive API keys see the forum (https://forum.rclone.org/t/rclone-has-been-banned- from-amazon-drive/2314).
If you happen to know anyone who works at Amazon then please ask them to re-instate rclone into the Amazon Drive developer program - thanks!
Configuration The initial setup for Amazon Drive involves getting a token from Amazon which you need to do in your browser.  rclone config walks you through it.
The configuration process for Amazon Drive may involve using an oauth proxy (https://github.com/ncw/oauthproxy).  This is used to keep the Amazon credentials out of the source code.  The proxy runs in Google's very secure App Engine environment and doesn't store any credentials which pass through it.
Since rclone doesn't currently have its own Amazon Drive credentials so you will either need to have your own client_id and client_secret with Amazon Drive, or use a third party oauth proxy in which case you will need to enter client_id, client_secret, auth_url and token_url.
Note also if you are not using Amazon's auth_url and token_url, (ie you filled in something for those) then if setting up on a remote machine you can only use the copying the config method of configuration (https://rclone.org/remote_setup/#configuring-by-copying-the-config-file) - rclone authorize will not work.
No remotes found - make a new one n) New remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config n/r/c/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / Amazon Drive \ "amazon cloud drive" [snip] Storage> amazon cloud drive Amazon Application Client Id - required. client_id> your client ID goes here Amazon Application Client Secret - required. client_secret> your client secret goes here Auth server URL - leave blank to use Amazon's. auth_url> Optional auth URL Token server url - leave blank to use Amazon's. token_url> Optional token URL Remote config Make sure your Redirect URL is set to "http://127.0.0.1:53682/" in your custom config. Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes n) No y/n> y If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code -------------------- [remote] client_id = your client ID goes here client_secret = your client secret goes here auth_url = Optional auth URL token_url = Optional token URL token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","refresh_token":"xxxxxxxxxxxxxxxxxx","expiry":"2015-09-06T16:07:39.658438471+01:00"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
See the remote setup docs (https://rclone.org/remote_setup/) for how to set it up on a machine with no Internet browser available.
Note that rclone runs a webserver on your local machine to collect the token as returned from Amazon.  This only runs from the moment it opens your browser to the moment you get back the verification code.  This is on http://127.0.0.1:53682/ and this it may require you to unblock it temporarily if you are running a host firewall.
List directories in top level of your Amazon Drive
List all the files in your Amazon Drive
To copy a local directory to an Amazon Drive directory called backup
Modified time and MD5SUMs Amazon Drive doesn't allow modification times to be changed via the API so these won't be accurate or used for syncing.
It does store MD5SUMs so for a more accurate sync, you can use the --checksum flag.
Restricted filename characters Character   Value   Replacement ──────────────────────────────── NUL         0x00         ␀ /           0x2F         ／
Deleting files Any files you delete with rclone will end up in the trash.  Amazon don't provide an API to permanently delete files, nor to empty the trash, so you will have to do that with one of Amazon's apps or via the Amazon Drive website.  As of November 17, 2016, files are automatically deleted by Amazon from the trash after 30 days.
Using with non .com Amazon accounts Let's say you usually use amazon.co.uk.  When you authenticate with rclone it will take you to an amazon.com page to log in.  Your amazon.co.uk email and password should work here just fine.
Standard options Here are the standard options specific to amazon cloud drive (Amazon Drive).
--acd-client-id OAuth Client Id.
Leave blank normally.
• Config: client_id
• Env Var: RCLONE_ACD_CLIENT_ID
--acd-client-secret OAuth Client Secret.
• Config: client_secret
• Env Var: RCLONE_ACD_CLIENT_SECRET
Advanced options Here are the advanced options specific to amazon cloud drive (Amazon Drive).
--acd-token OAuth Access Token as a JSON blob.
• Config: token
• Env Var: RCLONE_ACD_TOKEN
--acd-auth-url Auth server URL.
Leave blank to use the provider defaults.
• Config: auth_url
• Env Var: RCLONE_ACD_AUTH_URL
--acd-token-url Token server url.
• Config: token_url
• Env Var: RCLONE_ACD_TOKEN_URL
--acd-checkpoint Checkpoint for internal polling (debug).
• Config: checkpoint
• Env Var: RCLONE_ACD_CHECKPOINT
--acd-upload-wait-per-gb Additional time per GiB to wait after a failed complete upload to see if it appears.
Sometimes Amazon Drive gives an error when a file has been fully uploaded but the file appears anyway after a little while.  This happens sometimes for files over 1 GiB in size and nearly every time for files bigger than 10 GiB.  This parameter controls the time rclone waits for the file to appear.
The default value for this parameter is 3 minutes per GiB, so by default it will wait 3 minutes for every GiB uploaded to see if the file appears.
You can disable this feature by setting it to 0.  This may cause conflict errors as rclone retries the failed upload but the file will most likely appear correctly eventually.
These values were determined empirically by observing lots of uploads of big files for a range of file sizes.
Upload with the "-v" flag to see more info about what rclone is doing in this situation.
• Config: upload_wait_per_gb
• Env Var: RCLONE_ACD_UPLOAD_WAIT_PER_GB
• Type: Duration
• Default: 3m0s
--acd-templink-threshold Files >= this size will be downloaded via their tempLink.
Files this size or more will be downloaded via their "tempLink".  This is to work around a problem with Amazon Drive which blocks downloads of files bigger than about 10 GiB.  The default for this is 9 GiB which shouldn't need to be changed.
To download files above this threshold, rclone requests a "tempLink" which downloads the file through a temporary URL directly from the underlying S3 storage.
• Config: templink_threshold
• Env Var: RCLONE_ACD_TEMPLINK_THRESHOLD
• Type: SizeSuffix
• Default: 9Gi
--acd-encoding This sets the encoding for the backend.
• Env Var: RCLONE_ACD_ENCODING
• Default: Slash,InvalidUtf8,Dot
Limitations Note that Amazon Drive is case insensitive so you can't have a file called "Hello.doc" and one called "hello.doc".
Amazon Drive has rate limiting so you may notice errors in the sync (429 errors).  rclone will automatically retry the sync up to 3 times by default (see --retries flag) which should hopefully work around this problem.
Amazon Drive has an internal limit of file sizes that can be uploaded to the service.  This limit is not officially published, but all files larger than this will fail.
At the time of writing (Jan 2016) is in the area of 50 GiB per file. This means that larger files are likely to fail.
Unfortunately there is no way for rclone to see that this failure is because of file size, so it will retry the operation, as any other failure.  To avoid this problem, use --max-size 50000M option to limit the maximum size of uploaded files.  Note that --max-size does not split files into segments, it only ignores files over this size.
rclone about is not supported by the Amazon Drive backend.  Backends without this capability cannot determine free space for an rclone mount or use policy mfs (most free space) as a member of an rclone union remote.
Amazon S3 Storage Providers The S3 backend can be used with a number of different providers:
• AWS S3
Paths are specified as remote:bucket (or remote: for the lsd command.) You may put subdirectories in too, e.g.  remote:bucket/path/to/dir.
Once you have made a remote (see the provider specific section above) you can use it like this:
See all buckets
Make a new bucket
rclone mkdir remote:bucket
List the contents of a bucket
rclone ls remote:bucket
Sync /home/local/directory to the remote bucket, deleting any excess files in the bucket.
rclone sync -i /home/local/directory remote:bucket
Configuration Here is an example of making an s3 configuration for the AWS S3 provider. Most applies to the other providers as well, any differences are described below.
This will guide you through an interactive setup process.
Modified time The modified time is stored as metadata on the object as X-Amz-Meta-Mtime as floating point since the epoch, accurate to 1 ns.
If the modification time needs to be updated rclone will attempt to perform a server side copy to update the modification if the object can be copied in a single part.  In the case the object is larger than 5Gb or is in Glacier or Glacier Deep Archive storage the object will be uploaded rather than copied.
Note that reading this from the object takes an additional HEAD request as the metadata isn't returned in object listings.
Reducing costs Avoiding HEAD requests to read the modification time By default rclone will use the modification time of objects stored in S3 for syncing.  This is stored in object metadata which unfortunately takes an extra HEAD request to read which can be expensive (in time and money).
The modification time is used by default for all operations that require checking the time a file was last updated.  It allows rclone to treat the remote more like a true filesystem, but it is inefficient on S3 because it requires an extra API call to retrieve the metadata.
The extra API calls can be avoided when syncing (using rclone sync or rclone copy) in a few different ways, each with its own tradeoffs.
• --size-only
• Only checks the size of files.
• Uses no extra transactions.
• If the file doesn't change size then rclone won't detect it has changed.
• rclone sync --size-only /path/to/source s3:bucket
• --checksum
• Checks the size and MD5 checksum of files.
• The most accurate detection of changes possible.
• Will cause the source to read an MD5 checksum which, if it is a local disk, will cause lots of disk activity.
• If the source and destination are both S3 this is the recommended flag to use for maximum efficiency.
• rclone sync --checksum /path/to/source s3:bucket
• --update --use-server-modtime
• Modification time becomes the time the object was uploaded.
• For many operations this is sufficient to determine if it needs uploading.
• Using --update along with --use-server-modtime, avoids the extra API call and uploads files whose local modification time is newer than the time it was last uploaded.
• Files created with timestamps in the past will be missed by the sync.
• rclone sync --update --use-server-modtime /path/to/source s3:bucket
These flags can and should be used in combination with --fast-list - see below.
If using rclone mount or any command using the VFS (eg rclone serve) commands then you might want to consider using the VFS flag --no-modtime which will stop rclone reading the modification time for every object. You could also use --use-server-modtime if you are happy with the modification times of the objects being the time of upload.
Avoiding GET requests to read directory listings Rclone's default directory traversal is to process each directory individually.  This takes one API call per directory.  Using the --fast-list flag will read all info about the the objects into memory first using a smaller number of API calls (one per 1000 objects).  See the rclone docs (https://rclone.org/docs/#fast-list) for more details.
rclone sync --fast-list --checksum /path/to/source s3:bucket
--fast-list trades off API transactions for memory use.  As a rough guide rclone uses 1k of memory per object stored, so using --fast-list on a sync of a million objects will use roughly 1 GiB of RAM.
If you are only copying a small number of files into a big repository then using --no-traverse is a good idea.  This finds objects directly instead of through directory listings.  You can do a "top-up" sync very cheaply by using --max-age and --no-traverse to copy only recent files, eg
rclone copy --min-age 24h --no-traverse /path/to/source s3:bucket
You'd then do a full rclone sync less often.
Note that --fast-list isn't required in the top-up sync.
Avoiding HEAD requests after PUT By default rclone will HEAD every object it uploads.  It does this to check the object got uploaded correctly.
You can disable this with the --s3-no-head option - see there for more details.
Setting this flag increases the chance for undetected upload failures.
Hashes For small objects which weren't uploaded as multipart uploads (objects sized below --s3-upload-cutoff if uploaded with rclone) rclone uses the ETag: header as an MD5 checksum.
However for objects which were uploaded as multipart uploads or with server side encryption (SSE-AWS or SSE-C) the ETag header is no longer the MD5 sum of the data, so rclone adds an additional piece of metadata X-Amz-Meta-Md5chksum which is a base64 encoded MD5 hash (in the same format as is required for Content-MD5).
For large objects, calculating this hash can take some time so the addition of this hash can be disabled with --s3-disable-checksum.  This will mean that these objects do not have an MD5 checksum.
Cleanup If you run rclone cleanup s3:bucket then it will remove all pending multipart uploads older than 24 hours.  You can use the -i flag to see exactly what it will do.  If you want more control over the expiry date then run rclone backend cleanup s3:bucket -o max-age=1h to expire all uploads older than one hour.  You can use rclone backend list-multipart-uploads s3:bucket to see the pending multipart uploads.
Restricted filename characters S3 allows any valid UTF-8 string as a key.
Invalid UTF-8 bytes will be replaced (https://rclone.org/overview/#invalid-utf8), as they can't be used in XML.
The following characters are replaced since these are problematic when dealing with the REST API:
Character   Value   Replacement ──────────────────────────────── NUL         0x00         ␀ /           0x2F         ／
The encoding will also encode these file names as they don't seem to work with the SDK properly:
Multipart uploads rclone supports multipart uploads with S3 which means that it can upload files bigger than 5 GiB.
Note that files uploaded both with multipart upload and through crypt remotes do not have MD5 sums.
rclone switches from single part uploads to multipart uploads at the point specified by --s3-upload-cutoff.  This can be a maximum of 5 GiB and a minimum of 0 (ie always upload multipart files).
The chunk sizes used in the multipart upload are specified by --s3-chunk-size and the number of chunks uploaded concurrently is specified by --s3-upload-concurrency.
Multipart uploads will use --transfers * --s3-upload-concurrency * --s3-chunk-size extra memory.  Single part uploads to not use extra memory.
Single part transfers can be faster than multipart transfers or slower depending on your latency from S3 - the more latency, the more likely single part transfers will be faster.
Increasing --s3-upload-concurrency will increase throughput (8 would be a sensible value) and increasing --s3-chunk-size also increases throughput (16M would be sensible).  Increasing either of these will use more memory.  The default values are high enough to gain most of the possible performance without using too much memory.
Buckets and Regions With Amazon S3 you can list buckets (rclone lsd) using any region, but you can only access the content of a bucket from the region it was created in.  If you attempt to access a bucket from the wrong region, you will get an error, incorrect region, the bucket is not in 'XXX' region.
Authentication There are a number of ways to supply rclone with a set of AWS credentials, with and without using the environment.
The different authentication methods are tried in this order:
• Directly in the rclone configuration file (env_auth = false in the config file):
• access_key_id and secret_access_key are required.
• session_token can be optionally set when using AWS STS.
• Runtime configuration (env_auth = true in the config file):
• Export the following environment variables before running rclone:
• Access Key ID: AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY
• Secret Access Key: AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY
• Session Token: AWS_SESSION_TOKEN (optional)
• Or, use a named profile (https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple- profiles.html):
• Profile files are standard files used by AWS CLI tools
• By default it will use the profile in your home directory (e.g. ~/.aws/credentials on unix based systems) file and the "default" profile, to change set these environment variables:
• AWS_SHARED_CREDENTIALS_FILE to control which file.
• AWS_PROFILE to control which profile to use.
• Or, run rclone in an ECS task with an IAM role (AWS only).
• Or, run rclone on an EC2 instance with an IAM role (AWS only).
• Or, run rclone in an EKS pod with an IAM role that is associated with a service account (AWS only).
If none of these option actually end up providing rclone with AWS credentials then S3 interaction will be non-authenticated (see below).
S3 Permissions When using the sync subcommand of rclone the following minimum permissions are required to be available on the bucket being written to:
• ListBucket
• DeleteObject
• GetObject
• PutObject
• PutObjectACL
When using the lsd subcommand, the ListAllMyBuckets permission is required.
Example policy:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::USER_SID:user/USER_NAME" }, "Action": [ "s3:ListBucket", "s3:DeleteObject", "s3:GetObject", "s3:PutObject", "s3:PutObjectAcl" ], "Resource": [ "arn:aws:s3:::BUCKET_NAME/*", "arn:aws:s3:::BUCKET_NAME" ] }, { "Effect": "Allow", "Action": "s3:ListAllMyBuckets", "Resource": "arn:aws:s3:::*" } ] }
Notes on above:
1. This is a policy that can be used when creating bucket.  It assumes that USER_NAME has been created.
2. The Resource entry must include both resource ARNs, as one implies the bucket and the other implies the bucket's objects.
For reference, here's an Ansible script (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b) that will generate one or more buckets that will work with rclone sync.
Key Management System (KMS) If you are using server-side encryption with KMS then you must make sure rclone is configured with server_side_encryption = aws:kms otherwise you will find you can't transfer small objects - these will create checksum errors.
Glacier and Glacier Deep Archive You can upload objects using the glacier storage class or transition them to glacier using a lifecycle policy (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create- lifecycle.html).  The bucket can still be synced or copied into normally, but if rclone tries to access data from the glacier storage class you will see an error like below.
2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
In this case you need to restore (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived- objects.html) the object(s) in question before using rclone.
Note that rclone only speaks the S3 API it does not speak the Glacier Vault API, so rclone cannot directly access Glacier Vaults.
Standard options Here are the standard options specific to s3 (Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS).
--s3-provider Choose your S3 provider.
• Config: provider
• Env Var: RCLONE_S3_PROVIDER
• Examples:
• "AWS"
• Amazon Web Services (AWS) S3
• "Alibaba"
• Alibaba Cloud Object Storage System (OSS) formerly Aliyun
• "Ceph"
• Ceph Object Storage
• "DigitalOcean"
• Digital Ocean Spaces
• "Dreamhost"
• Dreamhost DreamObjects
• "IBMCOS"
• "Minio"
• Minio Object Storage
• "Netease"
• Netease Object Storage (NOS)
• "Scaleway"
• Scaleway Object Storage
• "SeaweedFS"
• SeaweedFS S3
• "StackPath"
• StackPath Object Storage
• "TencentCOS"
• "Wasabi"
• Wasabi Object Storage
• "Other"
• Any other S3 compatible provider
--s3-env-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
• Config: env_auth
• Env Var: RCLONE_S3_ENV_AUTH
• Type: bool
• Default: false
• "false"
• Enter AWS credentials in the next step.
• "true"
• Get AWS credentials from the environment (env vars or IAM).
--s3-access-key-id AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
• Config: access_key_id
• Env Var: RCLONE_S3_ACCESS_KEY_ID
--s3-secret-access-key AWS Secret Access Key (password).
• Config: secret_access_key
• Env Var: RCLONE_S3_SECRET_ACCESS_KEY
--s3-region Region to connect to.
• Config: region
• Env Var: RCLONE_S3_REGION
• "us-east-1"
• The default endpoint - a good choice if you are unsure.
• US Region, Northern Virginia, or Pacific Northwest.
• Leave location constraint empty.
• "us-east-2"
• US East (Ohio) Region.
• Needs location constraint us-east-2.
• "us-west-1"
• US West (Northern California) Region.
• Needs location constraint us-west-1.
• "us-west-2"
• US West (Oregon) Region.
• Needs location constraint us-west-2.
• "ca-central-1"
• Canada (Central) Region.
• Needs location constraint ca-central-1.
• "eu-west-1"
• EU (Ireland) Region.
• Needs location constraint EU or eu-west-1.
• "eu-west-2"
• EU (London) Region.
• Needs location constraint eu-west-2.
• "eu-west-3"
• EU (Paris) Region.
• Needs location constraint eu-west-3.
• "eu-north-1"
• EU (Stockholm) Region.
• Needs location constraint eu-north-1.
• "eu-south-1"
• EU (Milan) Region.
• Needs location constraint eu-south-1.
• "eu-central-1"
• EU (Frankfurt) Region.
• Needs location constraint eu-central-1.
• "ap-southeast-1"
• Asia Pacific (Singapore) Region.
• Needs location constraint ap-southeast-1.
• "ap-southeast-2"
• Asia Pacific (Sydney) Region.
• Needs location constraint ap-southeast-2.
• "ap-northeast-1"
• Asia Pacific (Tokyo) Region.
• Needs location constraint ap-northeast-1.
• "ap-northeast-2"
• Asia Pacific (Seoul).
• Needs location constraint ap-northeast-2.
• "ap-northeast-3"
• Asia Pacific (Osaka-Local).
• Needs location constraint ap-northeast-3.
• "ap-south-1"
• Asia Pacific (Mumbai).
• Needs location constraint ap-south-1.
• "ap-east-1"
• Asia Pacific (Hong Kong) Region.
• Needs location constraint ap-east-1.
• "sa-east-1"
• South America (Sao Paulo) Region.
• Needs location constraint sa-east-1.
• "me-south-1"
• Middle East (Bahrain) Region.
• Needs location constraint me-south-1.
• "af-south-1"
• Africa (Cape Town) Region.
• Needs location constraint af-south-1.
• "cn-north-1"
• China (Beijing) Region.
• Needs location constraint cn-north-1.
• "cn-northwest-1"
• China (Ningxia) Region.
• Needs location constraint cn-northwest-1.
• "us-gov-east-1"
• AWS GovCloud (US-East) Region.
• Needs location constraint us-gov-east-1.
• "us-gov-west-1"
• AWS GovCloud (US) Region.
• Needs location constraint us-gov-west-1.
• "nl-ams"
• Amsterdam, The Netherlands
• "fr-par"
• Paris, France
Leave blank if you are using an S3 clone and you don't have a region.
• ""
• Use this if unsure.
• Will use v4 signatures and an empty region.
• "other-v2-signature"
• Use this only if v4 signatures don't work.
• E.g.  pre Jewel/v10 CEPH.
--s3-endpoint Endpoint for S3 API.
Leave blank if using AWS to use the default endpoint for the region.
• Config: endpoint
• Env Var: RCLONE_S3_ENDPOINT
--s3-endpoint Endpoint for IBM COS S3 API.
Specify if using an IBM COS On Premise.
• "s3.us.cloud-object-storage.appdomain.cloud"
• US Cross Region Endpoint
• "s3.dal.us.cloud-object-storage.appdomain.cloud"
• US Cross Region Dallas Endpoint
• "s3.wdc.us.cloud-object-storage.appdomain.cloud"
• US Cross Region Washington DC Endpoint
• "s3.sjc.us.cloud-object-storage.appdomain.cloud"
• US Cross Region San Jose Endpoint
• "s3.private.us.cloud-object-storage.appdomain.cloud"
• US Cross Region Private Endpoint
• "s3.private.dal.us.cloud-object-storage.appdomain.cloud"
• US Cross Region Dallas Private Endpoint
• "s3.private.wdc.us.cloud-object-storage.appdomain.cloud"
• US Cross Region Washington DC Private Endpoint
• "s3.private.sjc.us.cloud-object-storage.appdomain.cloud"
• US Cross Region San Jose Private Endpoint
• "s3.us-east.cloud-object-storage.appdomain.cloud"
• US Region East Endpoint
• "s3.private.us-east.cloud-object-storage.appdomain.cloud"
• US Region East Private Endpoint
• "s3.us-south.cloud-object-storage.appdomain.cloud"
• US Region South Endpoint
• "s3.private.us-south.cloud-object-storage.appdomain.cloud"
• US Region South Private Endpoint
• "s3.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Endpoint
• "s3.fra.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Frankfurt Endpoint
• "s3.mil.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Milan Endpoint
• "s3.ams.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Amsterdam Endpoint
• "s3.private.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Private Endpoint
• "s3.private.fra.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Frankfurt Private Endpoint
• "s3.private.mil.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Milan Private Endpoint
• "s3.private.ams.eu.cloud-object-storage.appdomain.cloud"
• EU Cross Region Amsterdam Private Endpoint
• "s3.eu-gb.cloud-object-storage.appdomain.cloud"
• Great Britain Endpoint
• "s3.private.eu-gb.cloud-object-storage.appdomain.cloud"
• Great Britain Private Endpoint
• "s3.eu-de.cloud-object-storage.appdomain.cloud"
• EU Region DE Endpoint
• "s3.private.eu-de.cloud-object-storage.appdomain.cloud"
• EU Region DE Private Endpoint
• "s3.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional Endpoint
• "s3.tok.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional Tokyo Endpoint
• "s3.hkg.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional HongKong Endpoint
• "s3.seo.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional Seoul Endpoint
• "s3.private.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional Private Endpoint
• "s3.private.tok.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional Tokyo Private Endpoint
• "s3.private.hkg.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional HongKong Private Endpoint
• "s3.private.seo.ap.cloud-object-storage.appdomain.cloud"
• APAC Cross Regional Seoul Private Endpoint
• "s3.jp-tok.cloud-object-storage.appdomain.cloud"
• APAC Region Japan Endpoint
• "s3.private.jp-tok.cloud-object-storage.appdomain.cloud"
• APAC Region Japan Private Endpoint
• "s3.au-syd.cloud-object-storage.appdomain.cloud"
• APAC Region Australia Endpoint
• "s3.private.au-syd.cloud-object-storage.appdomain.cloud"
• APAC Region Australia Private Endpoint
• "s3.ams03.cloud-object-storage.appdomain.cloud"
• Amsterdam Single Site Endpoint
• "s3.private.ams03.cloud-object-storage.appdomain.cloud"
• Amsterdam Single Site Private Endpoint
• "s3.che01.cloud-object-storage.appdomain.cloud"
• Chennai Single Site Endpoint
• "s3.private.che01.cloud-object-storage.appdomain.cloud"
• Chennai Single Site Private Endpoint
• "s3.mel01.cloud-object-storage.appdomain.cloud"
• Melbourne Single Site Endpoint
• "s3.private.mel01.cloud-object-storage.appdomain.cloud"
• Melbourne Single Site Private Endpoint
• "s3.osl01.cloud-object-storage.appdomain.cloud"
• Oslo Single Site Endpoint
• "s3.private.osl01.cloud-object-storage.appdomain.cloud"
• Oslo Single Site Private Endpoint
• "s3.tor01.cloud-object-storage.appdomain.cloud"
• Toronto Single Site Endpoint
• "s3.private.tor01.cloud-object-storage.appdomain.cloud"
• Toronto Single Site Private Endpoint
• "s3.seo01.cloud-object-storage.appdomain.cloud"
• Seoul Single Site Endpoint
• "s3.private.seo01.cloud-object-storage.appdomain.cloud"
• Seoul Single Site Private Endpoint
• "s3.mon01.cloud-object-storage.appdomain.cloud"
• Montreal Single Site Endpoint
• "s3.private.mon01.cloud-object-storage.appdomain.cloud"
• Montreal Single Site Private Endpoint
• "s3.mex01.cloud-object-storage.appdomain.cloud"
• Mexico Single Site Endpoint
• "s3.private.mex01.cloud-object-storage.appdomain.cloud"
• Mexico Single Site Private Endpoint
• "s3.sjc04.cloud-object-storage.appdomain.cloud"
• San Jose Single Site Endpoint
• "s3.private.sjc04.cloud-object-storage.appdomain.cloud"
• San Jose Single Site Private Endpoint
• "s3.mil01.cloud-object-storage.appdomain.cloud"
• Milan Single Site Endpoint
• "s3.private.mil01.cloud-object-storage.appdomain.cloud"
• Milan Single Site Private Endpoint
• "s3.hkg02.cloud-object-storage.appdomain.cloud"
• Hong Kong Single Site Endpoint
• "s3.private.hkg02.cloud-object-storage.appdomain.cloud"
• Hong Kong Single Site Private Endpoint
• "s3.par01.cloud-object-storage.appdomain.cloud"
• Paris Single Site Endpoint
• "s3.private.par01.cloud-object-storage.appdomain.cloud"
• Paris Single Site Private Endpoint
• "s3.sng01.cloud-object-storage.appdomain.cloud"
• Singapore Single Site Endpoint
• "s3.private.sng01.cloud-object-storage.appdomain.cloud"
• Singapore Single Site Private Endpoint
--s3-endpoint Endpoint for OSS API.
• "oss-accelerate.aliyuncs.com"
• Global Accelerate
• "oss-accelerate-overseas.aliyuncs.com"
• Global Accelerate (outside mainland China)
• "oss-cn-hangzhou.aliyuncs.com"
• East China 1 (Hangzhou)
• "oss-cn-shanghai.aliyuncs.com"
• East China 2 (Shanghai)
• "oss-cn-qingdao.aliyuncs.com"
• North China 1 (Qingdao)
• "oss-cn-beijing.aliyuncs.com"
• North China 2 (Beijing)
• "oss-cn-zhangjiakou.aliyuncs.com"
• North China 3 (Zhangjiakou)
• "oss-cn-huhehaote.aliyuncs.com"
• North China 5 (Hohhot)
• "oss-cn-wulanchabu.aliyuncs.com"
• North China 6 (Ulanqab)
• "oss-cn-shenzhen.aliyuncs.com"
• South China 1 (Shenzhen)
• "oss-cn-heyuan.aliyuncs.com"
• South China 2 (Heyuan)
• "oss-cn-guangzhou.aliyuncs.com"
• South China 3 (Guangzhou)
• "oss-cn-chengdu.aliyuncs.com"
• West China 1 (Chengdu)
• "oss-cn-hongkong.aliyuncs.com"
• Hong Kong (Hong Kong)
• "oss-us-west-1.aliyuncs.com"
• US West 1 (Silicon Valley)
• "oss-us-east-1.aliyuncs.com"
• US East 1 (Virginia)
• "oss-ap-southeast-1.aliyuncs.com"
• Southeast Asia Southeast 1 (Singapore)
• "oss-ap-southeast-2.aliyuncs.com"
• Asia Pacific Southeast 2 (Sydney)
• "oss-ap-southeast-3.aliyuncs.com"
• Southeast Asia Southeast 3 (Kuala Lumpur)
• "oss-ap-southeast-5.aliyuncs.com"
• Asia Pacific Southeast 5 (Jakarta)
• "oss-ap-northeast-1.aliyuncs.com"
• Asia Pacific Northeast 1 (Japan)
• "oss-ap-south-1.aliyuncs.com"
• Asia Pacific South 1 (Mumbai)
• "oss-eu-central-1.aliyuncs.com"
• Central Europe 1 (Frankfurt)
• "oss-eu-west-1.aliyuncs.com"
• West Europe (London)
• "oss-me-east-1.aliyuncs.com"
• Middle East 1 (Dubai)
--s3-endpoint Endpoint for Scaleway Object Storage.
• "s3.nl-ams.scw.cloud"
• Amsterdam Endpoint
• "s3.fr-par.scw.cloud"
• Paris Endpoint
--s3-endpoint Endpoint for StackPath Object Storage.
• "s3.us-east-2.stackpathstorage.com"
• US East Endpoint
• "s3.us-west-1.stackpathstorage.com"
• US West Endpoint
• "s3.eu-central-1.stackpathstorage.com"
• EU Endpoint
--s3-endpoint Endpoint for Tencent COS API.
• "cos.ap-beijing.myqcloud.com"
• Beijing Region
• "cos.ap-nanjing.myqcloud.com"
• Nanjing Region
• "cos.ap-shanghai.myqcloud.com"
• Shanghai Region
• "cos.ap-guangzhou.myqcloud.com"
• Guangzhou Region
• "cos.ap-chengdu.myqcloud.com"
• Chengdu Region
• "cos.ap-chongqing.myqcloud.com"
• Chongqing Region
• "cos.ap-hongkong.myqcloud.com"
• Hong Kong (China) Region
• "cos.ap-singapore.myqcloud.com"
• Singapore Region
• "cos.ap-mumbai.myqcloud.com"
• Mumbai Region
• "cos.ap-seoul.myqcloud.com"
• Seoul Region
• "cos.ap-bangkok.myqcloud.com"
• Bangkok Region
• "cos.ap-tokyo.myqcloud.com"
• Tokyo Region
• "cos.na-siliconvalley.myqcloud.com"
• Silicon Valley Region
• "cos.na-ashburn.myqcloud.com"
• Virginia Region
• "cos.na-toronto.myqcloud.com"
• Toronto Region
• "cos.eu-frankfurt.myqcloud.com"
• Frankfurt Region
• "cos.eu-moscow.myqcloud.com"
• Moscow Region
• "cos.accelerate.myqcloud.com"
• Use Tencent COS Accelerate Endpoint
Required when using an S3 clone.
• "objects-us-east-1.dream.io"
• Dream Objects endpoint
• "nyc3.digitaloceanspaces.com"
• Digital Ocean Spaces New York 3
• "ams3.digitaloceanspaces.com"
• Digital Ocean Spaces Amsterdam 3
• "sgp1.digitaloceanspaces.com"
• Digital Ocean Spaces Singapore 1
• "localhost:8333"
• SeaweedFS S3 localhost
• "s3.wasabisys.com"
• Wasabi US East endpoint
• "s3.us-west-1.wasabisys.com"
• Wasabi US West endpoint
• "s3.eu-central-1.wasabisys.com"
• Wasabi EU Central endpoint
• "s3.ap-northeast-1.wasabisys.com"
• Wasabi AP Northeast endpoint
--s3-location-constraint Location constraint - must be set to match the Region.
Used when creating buckets only.
• Config: location_constraint
• Env Var: RCLONE_S3_LOCATION_CONSTRAINT
• Empty for US Region, Northern Virginia, or Pacific Northwest
• US East (Ohio) Region
• US West (Northern California) Region
• US West (Oregon) Region
• Canada (Central) Region
• EU (Ireland) Region
• EU (London) Region
• EU (Paris) Region
• EU (Stockholm) Region
• EU (Milan) Region
• "EU"
• EU Region
• Asia Pacific (Singapore) Region
• Asia Pacific (Sydney) Region
• Asia Pacific (Tokyo) Region
• Asia Pacific (Seoul) Region
• Asia Pacific (Osaka-Local) Region
• Asia Pacific (Mumbai) Region
• Asia Pacific (Hong Kong) Region
• South America (Sao Paulo) Region
• Middle East (Bahrain) Region
• Africa (Cape Town) Region
• China (Beijing) Region
• China (Ningxia) Region
• AWS GovCloud (US-East) Region
• AWS GovCloud (US) Region
--s3-location-constraint Location constraint - must match endpoint when using IBM Cloud Public.
For on-prem COS, do not make a selection from this list, hit enter.
• "us-standard"
• US Cross Region Standard
• "us-vault"
• US Cross Region Vault
• "us-cold"
• US Cross Region Cold
• "us-flex"
• US Cross Region Flex
• "us-east-standard"
• US East Region Standard
• "us-east-vault"
• US East Region Vault
• "us-east-cold"
• US East Region Cold
• "us-east-flex"
• US East Region Flex
• "us-south-standard"
• US South Region Standard
• "us-south-vault"
• US South Region Vault
• "us-south-cold"
• US South Region Cold
• "us-south-flex"
• US South Region Flex
• "eu-standard"
• EU Cross Region Standard
• "eu-vault"
• EU Cross Region Vault
• "eu-cold"
• EU Cross Region Cold
• "eu-flex"
• EU Cross Region Flex
• "eu-gb-standard"
• Great Britain Standard
• "eu-gb-vault"
• Great Britain Vault
• "eu-gb-cold"
• Great Britain Cold
• "eu-gb-flex"
• Great Britain Flex
• "ap-standard"
• APAC Standard
• "ap-vault"
• APAC Vault
• "ap-cold"
• APAC Cold
• "ap-flex"
• APAC Flex
• "mel01-standard"
• Melbourne Standard
• "mel01-vault"
• Melbourne Vault
• "mel01-cold"
• Melbourne Cold
• "mel01-flex"
• Melbourne Flex
• "tor01-standard"
• Toronto Standard
• "tor01-vault"
• Toronto Vault
• "tor01-cold"
• Toronto Cold
• "tor01-flex"
• Toronto Flex
Leave blank if not sure.  Used when creating buckets only.
--s3-acl Canned ACL used when creating buckets and storing or copying objects.
This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
Note that this ACL is applied when server-side copying objects as S3 doesn't copy the ACL from the source but rather writes a fresh one.
• Config: acl
• Env Var: RCLONE_S3_ACL
• "default"
• Owner gets Full_CONTROL.
• No one else has access rights (default).
• "private"
• Owner gets FULL_CONTROL.
• "public-read"
• The AllUsers group gets READ access.
• "public-read-write"
• The AllUsers group gets READ and WRITE access.
• Granting this on a bucket is generally not recommended.
• "authenticated-read"
• The AuthenticatedUsers group gets READ access.
• "bucket-owner-read"
• Object owner gets FULL_CONTROL.
• Bucket owner gets READ access.
• If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
• "bucket-owner-full-control"
• Both the object owner and the bucket owner get FULL_CONTROL over the object.
• This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise COS.
• This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise IBM COS.
• This acl is available on IBM Cloud (Infra), On-Premise IBM COS.
• Not supported on Buckets.
• This acl is available on IBM Cloud (Infra) and On-Premise IBM COS.
--s3-server-side-encryption The server-side encryption algorithm used when storing this object in S3.
• Config: server_side_encryption
• Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION
• None
• "AES256"
• AES256
• "aws:kms"
• aws:kms
--s3-sse-kms-key-id If using KMS ID you must provide the ARN of Key.
• Config: sse_kms_key_id
• Env Var: RCLONE_S3_SSE_KMS_KEY_ID
• "arn:aws:kms:us-east-1:*"
• arn:aws:kms:*
--s3-storage-class The storage class to use when storing new objects in S3.
• Config: storage_class
• Env Var: RCLONE_S3_STORAGE_CLASS
• Default
• "STANDARD"
• Standard storage class
• "REDUCED_REDUNDANCY"
• Reduced redundancy storage class
• "STANDARD_IA"
• Standard Infrequent Access storage class
• "ONEZONE_IA"
• One Zone Infrequent Access storage class
• "GLACIER"
• Glacier storage class
• "DEEP_ARCHIVE"
• Glacier Deep Archive storage class
• "INTELLIGENT_TIERING"
• Intelligent-Tiering storage class
--s3-storage-class The storage class to use when storing new objects in OSS.
• Archive storage mode
• Infrequent access storage mode
--s3-storage-class The storage class to use when storing new objects in Tencent COS.
• "ARCHIVE"
• Default.
• The Standard class for any upload.
• Suitable for on-demand content like streaming or CDN.
• Archived storage.
• Prices are lower, but it needs to be restored first to be accessed.
Advanced options Here are the advanced options specific to s3 (Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS).
--s3-bucket-acl Canned ACL used when creating buckets.
Note that this ACL is applied when only when creating buckets.  If it isn't set then "acl" is used instead.
• Config: bucket_acl
• Env Var: RCLONE_S3_BUCKET_ACL
--s3-requester-pays Enables requester pays option when interacting with S3 bucket.
• Config: requester_pays
• Env Var: RCLONE_S3_REQUESTER_PAYS
--s3-sse-customer-algorithm If using SSE-C, the server-side encryption algorithm used when storing this object in S3.
• Config: sse_customer_algorithm
• Env Var: RCLONE_S3_SSE_CUSTOMER_ALGORITHM
--s3-sse-customer-key If using SSE-C you must provide the secret encryption key used to encrypt/decrypt your data.
• Config: sse_customer_key
• Env Var: RCLONE_S3_SSE_CUSTOMER_KEY
--s3-sse-customer-key-md5 If using SSE-C you may provide the secret encryption key MD5 checksum (optional).
If you leave it blank, this is calculated automatically from the sse_customer_key provided.
• Config: sse_customer_key_md5
• Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_MD5
--s3-upload-cutoff Cutoff for switching to chunked upload.
Any files larger than this will be uploaded in chunks of chunk_size.  The minimum is 0 and the maximum is 5 GiB.
• Config: upload_cutoff
• Env Var: RCLONE_S3_UPLOAD_CUTOFF
• Default: 200Mi
--s3-chunk-size Chunk size to use for uploading.
When uploading files larger than upload_cutoff or files with unknown size (e.g.  from "rclone rcat" or uploaded with "rclone mount" or google photos or google docs) they will be uploaded as multipart uploads using this chunk size.
Note that "--s3-upload-concurrency" chunks of this size are buffered in memory per transfer.
If you are transferring large files over high-speed links and you have enough memory, then increasing this will speed up the transfers.
Rclone will automatically increase the chunk size when uploading a large file of known size to stay below the 10,000 chunks limit.
Files of unknown size are uploaded with the configured chunk_size.  Since the default chunk size is 5 MiB and there can be at most 10,000 chunks, this means that by default the maximum size of a file you can stream upload is 48 GiB.  If you wish to stream upload larger files then you will need to increase chunk_size.
• Config: chunk_size
• Env Var: RCLONE_S3_CHUNK_SIZE
• Default: 5Mi
--s3-max-upload-parts Maximum number of parts in a multipart upload.
This option defines the maximum number of multipart chunks to use when doing a multipart upload.
This can be useful if a service does not support the AWS S3 specification of 10,000 chunks.
Rclone will automatically increase the chunk size when uploading a large file of a known size to stay below this number of chunks limit.
• Config: max_upload_parts
• Env Var: RCLONE_S3_MAX_UPLOAD_PARTS
• Type: int
• Default: 10000
--s3-copy-cutoff Cutoff for switching to multipart copy.
Any files larger than this that need to be server-side copied will be copied in chunks of this size.
The minimum is 0 and the maximum is 5 GiB.
• Config: copy_cutoff
• Env Var: RCLONE_S3_COPY_CUTOFF
• Default: 4.656Gi
--s3-disable-checksum Don't store MD5 checksum with object metadata.
Normally rclone will calculate the MD5 checksum of the input before uploading it so it can add it to metadata on the object.  This is great for data integrity checking but can cause long delays for large files to start uploading.
• Config: disable_checksum
• Env Var: RCLONE_S3_DISABLE_CHECKSUM
--s3-shared-credentials-file Path to the shared credentials file.
If env_auth = true then rclone can use a shared credentials file.
If this variable is empty rclone will look for the "AWS_SHARED_CREDENTIALS_FILE" env variable.  If the env value is empty it will default to the current user's home directory.
Linux/OSX: "$HOME/.aws/credentials" Windows:   "%USERPROFILE%\.aws\credentials"
• Config: shared_credentials_file
• Env Var: RCLONE_S3_SHARED_CREDENTIALS_FILE
--s3-profile Profile to use in the shared credentials file.
If env_auth = true then rclone can use a shared credentials file.  This variable controls which profile is used in that file.
If empty it will default to the environment variable "AWS_PROFILE" or "default" if that environment variable is also not set.
• Config: profile
• Env Var: RCLONE_S3_PROFILE
--s3-session-token An AWS session token.
• Config: session_token
• Env Var: RCLONE_S3_SESSION_TOKEN
--s3-upload-concurrency Concurrency for multipart uploads.
This is the number of chunks of the same file that are uploaded concurrently.
If you are uploading small numbers of large files over high-speed links and these uploads do not fully utilize your bandwidth, then increasing this may help to speed up the transfers.
• Config: upload_concurrency
• Env Var: RCLONE_S3_UPLOAD_CONCURRENCY
• Default: 4
--s3-force-path-style If true use path style access if false use virtual hosted style.
If this is true (the default) then rclone will use path style access, if false then rclone will use virtual path style.  See the AWS S3 docs (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access- bucket-intro) for more info.
Some providers (e.g.  AWS, Aliyun OSS, Netease COS, or Tencent COS) require this set to false - rclone will do this automatically based on the provider setting.
• Config: force_path_style
• Env Var: RCLONE_S3_FORCE_PATH_STYLE
• Default: true
--s3-v2-auth If true use v2 authentication.
If this is false (the default) then rclone will use v4 authentication. If it is set then rclone will use v2 authentication.
Use this only if v4 signatures don't work, e.g.  pre Jewel/v10 CEPH.
• Config: v2_auth
• Env Var: RCLONE_S3_V2_AUTH
--s3-use-accelerate-endpoint If true use the AWS S3 accelerated endpoint.
See: AWS S3 Transfer acceleration (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration- examples.html)
• Config: use_accelerate_endpoint
• Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT
--s3-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery.
It should be set to true for resuming uploads across different sessions.
WARNING: Storing parts of an incomplete multipart upload counts towards space usage on S3 and will add additional costs if not cleaned up.
• Config: leave_parts_on_error
• Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR
--s3-list-chunk Size of listing chunk (response list for each ListObject S3 request).
This option is also known as "MaxKeys", "max-items", or "page-size" from the AWS S3 specification.  Most services truncate the response list to 1000 objects even if requested more than that.  In AWS S3 this is a global maximum and cannot be changed, see AWS S3 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html).  In Ceph, this can be increased with the "rgw list buckets max chunk" option.
• Config: list_chunk
• Env Var: RCLONE_S3_LIST_CHUNK
• Default: 1000
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it.
This can be useful when trying to minimise the number of transactions rclone does if you know the bucket exists already.
It can also be needed if the user you are using does not have bucket creation permissions.  Before v1.52.0 this would have passed silently due to a bug.
• Config: no_check_bucket
• Env Var: RCLONE_S3_NO_CHECK_BUCKET
--s3-no-head If set, don't HEAD uploaded objects to check integrity.
This can be useful when trying to minimise the number of transactions rclone does.
Setting it means that if rclone receives a 200 OK message after uploading an object with PUT then it will assume that it got uploaded properly.
In particular it will assume:
• the metadata, including modtime, storage class and content type was as uploaded
• the size was as uploaded
It reads the following items from the response for a single part PUT:
• the MD5SUM
• The uploaded date
For multipart uploads these items aren't read.
If an source object of unknown length is uploaded then rclone will do a HEAD request.
Setting this flag increases the chance for undetected upload failures, in particular an incorrect size, so it isn't recommended for normal operation.  In practice the chance of an undetected upload failure is very small even with this flag.
• Config: no_head
• Env Var: RCLONE_S3_NO_HEAD
--s3-no-head-object If set, do not do HEAD before GET when getting objects.
• Config: no_head_object
• Env Var: RCLONE_S3_NO_HEAD_OBJECT
--s3-encoding This sets the encoding for the backend.
• Env Var: RCLONE_S3_ENCODING
--s3-memory-pool-flush-time How often internal memory buffer pools will be flushed.
Uploads which requires additional buffers (f.e multipart) will use memory pool for allocations.  This option controls how often unused buffers will be removed from the pool.
• Config: memory_pool_flush_time