neulab/tldr huggingface dataset:
Note that once blocked, the use of other tools (such as megacmd) is not a sure workaround: following megacmd login times have been observed in succession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min. Web access looks unaffected though.
Investigation is continuing in relation to workarounds based on timeouts, pacers, retrials and tpslimits - if you discover something relevant, please post on the forum.
So, if rclone was working nicely and suddenly you are unable to log-in and you are sure the user and the password are correct, likely you have got the remote blocked for a while.
Standard options Here are the standard options specific to mega (Mega).
--mega-user User name.
• Env Var: RCLONE_MEGA_USER
--mega-pass Password.
• Env Var: RCLONE_MEGA_PASS
Advanced options Here are the advanced options specific to mega (Mega).
--mega-debug Output more debug from Mega.
If this flag is set (along with -vv) it will print further debugging information from the mega backend.
• Config: debug
• Env Var: RCLONE_MEGA_DEBUG
--mega-hard-delete Delete files permanently rather than putting them into the trash.
Normally the mega backend will put all deletions into the trash rather than permanently deleting them.  If you specify this then rclone will permanently delete objects instead.
• Env Var: RCLONE_MEGA_HARD_DELETE
--mega-encoding This sets the encoding for the backend.
• Env Var: RCLONE_MEGA_ENCODING
Limitations This backend uses the go-mega go library (https://github.com/t3rm1n4l/go- mega) which is an opensource go library implementing the Mega API.  There doesn't appear to be any documentation for the mega protocol beyond the mega C++ SDK (https://github.com/meganz/sdk) source code so there are likely quite a few errors still remaining in this library.
Mega allows duplicate files which may confuse rclone.
Memory The memory backend is an in RAM backend.  It does not persist its data - use the local backend for that.
The memory backend behaves like a bucket based remote (e.g.  like s3). Because it has no parameters you can just use it with the :memory: remote name.
Configuration You can configure it as a remote like this with rclone config too if you want to:
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Memory \ "memory" [snip] Storage> memory ** See help for memory backend at: https://rclone.org/memory/ **
-------------------- [remote] type = memory -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
Because the memory backend isn't persistent it is most useful for testing or with an rclone server or rclone mount, e.g.
rclone mount :memory: /mnt/tmp rclone serve webdav :memory: rclone serve sftp :memory:
Modified time and hashes The memory backend supports MD5 hashes and modification times accurate to 1 nS.
Restricted filename characters The memory backend replaces the default restricted characters set (https://rclone.org/overview/#restricted-characters).
Microsoft Azure Blob Storage Paths are specified as remote:container (or remote: for the lsd command.) You may put subdirectories in too, e.g.  remote:container/path/to/dir.
Configuration Here is an example of making a Microsoft Azure Blob Storage configuration.  For a remote called remote.  First run:
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / Microsoft Azure Blob Storage \ "azureblob" [snip] Storage> azureblob Storage Account Name account> account_name Storage Account Key key> base64encodedkey== Endpoint for the service - leave blank normally. endpoint> Remote config -------------------- [remote] account = account_name key = base64encodedkey== endpoint = -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
See all containers
Make a new container
rclone mkdir remote:container
List the contents of a container
rclone ls remote:container
Sync /home/local/directory to the remote container, deleting any excess files in the container.
rclone sync -i /home/local/directory remote:container
Modified time The modified time is stored as metadata on the object with the mtime key. It is stored using RFC3339 Format time with nanosecond precision.  The metadata is supplied during directory listings so there is no overhead to using it.
Character   Value   Replacement ──────────────────────────────── /           0x2F         ／ \           0x5C         ＼
Character   Value   Replacement ──────────────────────────────── .           0x2E         ．
Hashes MD5 hashes are stored with blobs.  However blobs that were uploaded in chunks only have an MD5 if the source remote was capable of MD5 hashes, e.g.  the local disk.
Authenticating with Azure Blob Storage Rclone has 3 ways of authenticating with Azure Blob Storage:
Account and Key This is the most straight forward and least flexible way.  Just fill in the account and key lines and leave the rest blank.
SAS URL This can be an account level SAS URL or container level SAS URL.
To use it leave account, key blank and fill in sas_url.
An account level SAS URL or container level SAS URL can be obtained from the Azure portal or the Azure Storage Explorer.  To get a container level SAS URL right click on a container in the Azure Blob explorer in the Azure portal.
If you use a container level SAS URL, rclone operations are permitted only on a particular container, e.g.
rclone ls azureblob:container
You can also list the single container from the root.  This will only show the container specified by the SAS URL.
$ rclone lsd azureblob: container/
Note that you can't see or access any other containers - this will fail
rclone ls azureblob:othercontainer
Container level SAS URLs are useful for temporarily allowing third parties access to a single container or putting credentials into an untrusted environment such as a CI build server.
Standard options Here are the standard options specific to azureblob (Microsoft Azure Blob Storage).
--azureblob-account Storage Account Name.
Leave blank to use SAS URL or Emulator.
• Env Var: RCLONE_AZUREBLOB_ACCOUNT
--azureblob-service-principal-file Path to file containing credentials for use with a service principal.
Leave blank normally.  Needed only if you want to use a service principal instead of interactive login.
$ az ad sp create-for-rbac --name "<name>" \ --role "Storage Blob Data Owner" \ --scopes "/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default/containers/<container>" \ > azure-principal.json
See "Create an Azure service principal" (https://docs.microsoft.com/en- us/cli/azure/create-an-azure-service-principal-azure-cli) and "Assign an Azure role for access to blob data" (https://docs.microsoft.com/en- us/azure/storage/common/storage-auth-aad-rbac-cli) pages for more details.
• Config: service_principal_file
• Env Var: RCLONE_AZUREBLOB_SERVICE_PRINCIPAL_FILE
--azureblob-key Storage Account Key.
• Env Var: RCLONE_AZUREBLOB_KEY
--azureblob-sas-url SAS URL for container level access only.
Leave blank if using account/key or Emulator.
• Config: sas_url
• Env Var: RCLONE_AZUREBLOB_SAS_URL
--azureblob-use-msi Use a managed service identity to authenticate (only works in Azure).
When true, use a managed service identity (https://docs.microsoft.com/en- us/azure/active-directory/managed-identities-azure-resources/) to authenticate to Azure Storage instead of a SAS token or account key.
If the VM(SS) on which this program is running has a system-assigned identity, it will be used by default.  If the resource has no system-assigned but exactly one user-assigned identity, the user-assigned identity will be used by default.  If the resource has multiple user-assigned identities, the identity to use must be explicitly specified using exactly one of the msi_object_id, msi_client_id, or msi_mi_res_id parameters.
• Config: use_msi
• Env Var: RCLONE_AZUREBLOB_USE_MSI
--azureblob-use-emulator Uses local storage emulator if provided as 'true'.
Leave blank if using real azure storage endpoint.
• Config: use_emulator
• Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
Advanced options Here are the advanced options specific to azureblob (Microsoft Azure Blob Storage).
--azureblob-msi-object-id Object ID of the user-assigned MSI to use, if any.
Leave blank if msi_client_id or msi_mi_res_id specified.
• Config: msi_object_id
• Env Var: RCLONE_AZUREBLOB_MSI_OBJECT_ID
--azureblob-msi-client-id Object ID of the user-assigned MSI to use, if any.
Leave blank if msi_object_id or msi_mi_res_id specified.
• Config: msi_client_id
• Env Var: RCLONE_AZUREBLOB_MSI_CLIENT_ID
--azureblob-msi-mi-res-id Azure resource ID of the user-assigned MSI to use, if any.
Leave blank if msi_client_id or msi_object_id specified.
• Config: msi_mi_res_id
• Env Var: RCLONE_AZUREBLOB_MSI_MI_RES_ID
--azureblob-endpoint Endpoint for the service.
• Env Var: RCLONE_AZUREBLOB_ENDPOINT
--azureblob-upload-cutoff Cutoff for switching to chunked upload (<= 256 MiB) (deprecated).
• Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
--azureblob-chunk-size Upload chunk size (<= 100 MiB).
Note that this is stored in memory and there may be up to "--transfers" chunks stored at once in memory.
• Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
• Default: 4Mi
--azureblob-list-chunk Size of blob list.
This sets the number of blobs requested in each listing chunk.  Default is the maximum, 5000.  "List blobs" requests are permitted 2 minutes per megabyte to complete.  If an operation is taking longer than 2 minutes per megabyte on average, it will time out ( source (https://docs.microsoft.com/en-us/rest/api/storageservices/setting- timeouts-for-blob-service-operations#exceptions-to-default-timeout- interval) ).  This can be used to limit the number of blobs items to return, to avoid the time out.
• Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
• Default: 5000
--azureblob-access-tier Access tier of blob: hot, cool or archive.
Archived blobs can be restored by setting access tier to hot or cool. Leave blank if you intend to use default access tier, which is set at account level
If there is no "access tier" specified, rclone doesn't apply any tier. rclone performs "Set Tier" operation on blobs while uploading, if objects are not modified, specifying "access tier" to new one will have no effect.  If blobs are in "archive tier" at remote, trying to perform data transfer operations from remote will not be allowed.  User should first restore by tiering blob to "Hot" or "Cool".
• Config: access_tier
• Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
--azureblob-archive-tier-delete Delete archive tier blobs before overwriting.
Archive tier blobs cannot be updated.  So without this flag, if you attempt to update an archive tier blob, then rclone will produce the error:
can't update archive tier blob without --azureblob-archive-tier-delete
With this flag set then before rclone attempts to overwrite an archive tier blob, it will delete the existing blob before uploading its replacement.  This has the potential for data loss if the upload fails (unlike updating a normal blob) and also may cost more since deleting archive tier blobs early may be chargable.
• Config: archive_tier_delete
• Env Var: RCLONE_AZUREBLOB_ARCHIVE_TIER_DELETE
--azureblob-disable-checksum Don't store MD5 checksum with object metadata.
• Env Var: RCLONE_AZUREBLOB_DISABLE_CHECKSUM
--azureblob-memory-pool-flush-time How often internal memory buffer pools will be flushed.
• Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_FLUSH_TIME
--azureblob-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool.
• Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_USE_MMAP
--azureblob-encoding This sets the encoding for the backend.
• Env Var: RCLONE_AZUREBLOB_ENCODING
• Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8
--azureblob-public-access Public access level of a container: blob or container.
• Config: public_access
• Env Var: RCLONE_AZUREBLOB_PUBLIC_ACCESS
• The container and its blobs can be accessed only with an authorized request.
• It's a default value.
• "blob"
• Blob data within this container can be read via anonymous request.
• "container"
• Allow full public read access for container and blob data.
--azureblob-no-head-object If set, do not do HEAD before GET when getting objects.
• Env Var: RCLONE_AZUREBLOB_NO_HEAD_OBJECT
Limitations MD5 sums are only uploaded with chunked files if the source has an MD5 sum.  This will always be the case for a local to azure copy.
rclone about is not supported by the Microsoft Azure Blob storage backend.  Backends without this capability cannot determine free space for an rclone mount or use policy mfs (most free space) as a member of an rclone union remote.
Azure Storage Emulator Support You can test rclone with storage emulator locally, to do this make sure azure storage emulator installed locally and set up a new remote with rclone config follow instructions described in introduction, set use_emulator config as true, you do not need to provide default account name or key if using emulator.
Microsoft OneDrive Paths are specified as remote:path
Configuration The initial setup for OneDrive involves getting a token from Microsoft which you need to do in your browser.  rclone config walks you through it.
e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> n name> remote Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Microsoft OneDrive \ "onedrive" [snip] Storage> onedrive Microsoft App Client Id Leave blank normally. Enter a string value. Press Enter for the default (""). client_id> Microsoft App Client Secret Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> Edit advanced config? (y/n) y) Yes n) No y/n> n Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes n) No y/n> y If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code Choose a number from below, or type in an existing value 1 / OneDrive Personal or Business \ "onedrive" 2 / Sharepoint site \ "sharepoint" 3 / Type in driveID \ "driveid" 4 / Type in SiteID \ "siteid" 5 / Search a Sharepoint site \ "search" Your choice> 1 Found 1 drives, please select the one you want to use: 0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk Chose drive to use:> 0 Found drive 'root' of type 'business', URL: https://org-my.sharepoint.com/personal/you/Documents Is that okay? y) Yes n) No y/n> y -------------------- [remote] type = onedrive token = {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018-08-26T22:39:52.486512262+08:00"} drive_id = b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk drive_type = business -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
Note that rclone runs a webserver on your local machine to collect the token as returned from Microsoft.  This only runs from the moment it opens your browser to the moment you get back the verification code. This is on http://127.0.0.1:53682/ and this it may require you to unblock it temporarily if you are running a host firewall.
List directories in top level of your OneDrive
List all the files in your OneDrive
To copy a local directory to an OneDrive directory called backup
Getting your own Client ID and Key You can use your own Client ID if the default (client_id left blank) one doesn't work for you or you see lots of throttling.  The default Client ID and Key is shared by all rclone users when performing requests.
If you are having problems with them (E.g., seeing a lot of throttling), you can get your own Client ID and Key by following the steps below:
1. Open https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade and then click New registration.
2. Enter a name for your app, choose account type Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox), select Web in Redirect URI, then type (do not copy and paste) http://localhost:53682/ and click Register.  Copy and keep the Application (client) ID under the app name for later use.
3. Under manage select Certificates & secrets, click New client secret. Enter a description (can be anything) and set Expires to 24 months. Copy and keep that secret Value for later use (you won't be able to see this value afterwards).
4. Under manage select API permissions, click Add a permission and select Microsoft Graph then select delegated permissions.
5. Search and select the following permissions: Files.Read, Files.ReadWrite, Files.Read.All, Files.ReadWrite.All, offline_access, User.Read.  Once selected click Add permissions at the bottom.
Now the application is complete.  Run rclone config to create or edit a OneDrive remote.  Supply the app ID and password as Client ID and Secret, respectively.  rclone will walk you through the remaining steps.
Modification time and hashes OneDrive allows modification times to be set on objects accurate to 1 second.  These will be used to detect whether objects need syncing or not.
OneDrive personal supports SHA1 type hashes.  OneDrive for business and Sharepoint Server support QuickXorHash (https://docs.microsoft.com/en- us/onedrive/developer/code-snippets/quickxorhash).
For all types of OneDrive you can use the --checksum flag.
File names can also not begin with the following characters.  These only get replaced if they are the first character in the name:
Character   Value   Replacement ──────────────────────────────── SP          0x20         ␠ ~           0x7E         ～
Deleting files Any files you delete with rclone will end up in the trash.  Microsoft doesn't provide an API to permanently delete files, nor to empty the trash, so you will have to do that with one of Microsoft's apps or via the OneDrive website.
Standard options Here are the standard options specific to onedrive (Microsoft OneDrive).
--onedrive-client-id OAuth Client Id.
• Env Var: RCLONE_ONEDRIVE_CLIENT_ID
--onedrive-client-secret OAuth Client Secret.
• Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
--onedrive-region Choose national cloud region for OneDrive.
• Env Var: RCLONE_ONEDRIVE_REGION
• Default: "global"
• "global"
• Microsoft Cloud Global
• Microsoft Cloud for US Government
• "de"
• Microsoft Cloud Germany
• "cn"
• Azure and Office 365 operated by 21Vianet in China
Advanced options Here are the advanced options specific to onedrive (Microsoft OneDrive).
--onedrive-token OAuth Access Token as a JSON blob.
• Env Var: RCLONE_ONEDRIVE_TOKEN
--onedrive-auth-url Auth server URL.
• Env Var: RCLONE_ONEDRIVE_AUTH_URL
--onedrive-token-url Token server url.
• Env Var: RCLONE_ONEDRIVE_TOKEN_URL
--onedrive-chunk-size Chunk size to upload files with - must be multiple of 320k (327,680 bytes).
Above this size files will be chunked - must be multiple of 320k (327,680 bytes) and should not exceed 250M (262,144,000 bytes) else you may encounter "Microsoft.SharePoint.Client.InvalidClientQueryException: The request message is too big." Note that the chunks will be buffered into memory.
• Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
--onedrive-drive-id The ID of the drive to use.
• Config: drive_id
• Env Var: RCLONE_ONEDRIVE_DRIVE_ID
--onedrive-drive-type The type of the drive (personal | business | documentLibrary).
• Config: drive_type
• Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
--onedrive-expose-onenote-files Set to make OneNote files show up in directory listings.
By default rclone will hide OneNote files in directory listings because operations like "Open" and "Update" won't work on them.  But this behaviour may also prevent you from deleting them.  If you want to delete OneNote files or otherwise want them to show up in directory listing, set this option.
• Config: expose_onenote_files
• Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
--onedrive-server-side-across-configs Allow server-side operations (e.g.  copy) to work across different onedrive configs.
This will only work if you are copying between two OneDrive Personal drives AND the files to copy are already shared between them.  In other cases, rclone will fall back to normal copy (which will be slightly slower).
• Env Var: RCLONE_ONEDRIVE_SERVER_SIDE_ACROSS_CONFIGS
--onedrive-list-chunk Size of listing chunk.
• Env Var: RCLONE_ONEDRIVE_LIST_CHUNK
--onedrive-no-versions Remove all versions on modifying operations.
Onedrive for business creates versions when rclone uploads new files overwriting an existing one and when it sets the modification time.
These versions take up space out of the quota.
This flag checks for versions after file upload and setting modification time and removes all but the last version.
NB Onedrive personal can't currently delete versions so don't use this flag there.
• Env Var: RCLONE_ONEDRIVE_NO_VERSIONS
--onedrive-link-scope Set the scope of the links created by the link command.
• Config: link_scope
• Env Var: RCLONE_ONEDRIVE_LINK_SCOPE
• Default: "anonymous"
• "anonymous"
• Anyone with the link has access, without needing to sign in.
• This may include people outside of your organization.
• Anonymous link support may be disabled by an administrator.
• "organization"
• Anyone signed into your organization (tenant) can use the link to get access.
• Only available in OneDrive for Business and SharePoint.
--onedrive-link-type Set the type of the links created by the link command.
• Config: link_type
• Env Var: RCLONE_ONEDRIVE_LINK_TYPE
• Default: "view"
• "view"
• Creates a read-only link to the item.
• "edit"
• Creates a read-write link to the item.
• "embed"
• Creates an embeddable link to the item.
--onedrive-link-password Set the password for links created by the link command.
At the time of writing this only works with OneDrive personal paid accounts.
• Config: link_password
• Env Var: RCLONE_ONEDRIVE_LINK_PASSWORD
--onedrive-encoding This sets the encoding for the backend.
• Env Var: RCLONE_ONEDRIVE_ENCODING
• Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot
Limitations If you don't use rclone for 90 days the refresh token will expire.  This will result in authorization problems.  This is easy to fix by running the rclone config reconnect remote: command to get a new token and refresh token.
Naming Note that OneDrive is case insensitive so you can't have a file called "Hello.doc" and one called "hello.doc".
There are quite a few characters that can't be in OneDrive file names. These can't occur on Windows platforms, but on non-Windows platforms they are common.  Rclone will map these names to and from an identical looking unicode equivalent.  For example if a file has a ? in it will be mapped to ？ instead.
File sizes The largest allowed file size is 250 GiB for both OneDrive Personal and OneDrive for Business (Updated 13 Jan 2021) (https://support.microsoft.com/en-us/office/invalid-file-names-and-file- types-in-onedrive-and- sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa?ui=en-us&rs=en- us&ad=us#individualfilesize).
Path length The entire path, including the file name, must contain fewer than 400 characters for OneDrive, OneDrive for Business and SharePoint Online.  If you are encrypting file and folder names with rclone, you may want to pay attention to this limitation because the encrypted names are typically longer than the original ones.
Number of files OneDrive seems to be OK with at least 50,000 files in a folder, but at 100,000 rclone will get errors listing the directory like couldn’t list files: UnknownError:.  See #2707 (https://github.com/rclone/rclone/issues/2707) for more info.
An official document about the limitations for different types of OneDrive can be found here (https://support.office.com/en- us/article/invalid-file-names-and-file-types-in-onedrive-onedrive-for- business-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
Versions Every change in a file OneDrive causes the service to create a new version of the the file.  This counts against a users quota.  For example changing the modification time of a file creates a second version, so the file apparently uses twice the space.
For example the copy command is affected by this as rclone copies the file and then afterwards sets the modification time to match the source file which uses another version.
You can use the rclone cleanup command (see below) to remove all old versions.
Or you can set the no_versions parameter to true and rclone will remove versions after operations which create new versions.  This takes extra transactions so only enable it if you need it.
Note At the time of writing Onedrive Personal creates versions (but not for setting the modification time) but the API for removing them returns "API not found" so cleanup and no_versions should not be used on Onedrive Personal.
Disabling versioning Starting October 2018, users will no longer be able to disable versioning by default.  This is because Microsoft has brought an update (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New- Updates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390) to the mechanism.  To change this new default setting, a PowerShell command is required to be run by a SharePoint admin.  If you are an admin, you can run these commands in PowerShell to change that setting:
1. Install-Module -Name Microsoft.Online.SharePoint.PowerShell (in case you haven't installed this already)
2. Import-Module Microsoft.Online.SharePoint.PowerShell -DisableNameChecking
3. Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Credential YOU@YOURSITE.COM (replacing YOURSITE, YOU, YOURSITE.COM with the actual values; this will prompt for your credentials)
4. Set-SPOTenant -EnableMinimumVersionRequirement $False
5. Disconnect-SPOService (to disconnect from the server)
Below are the steps for normal users to disable versioning. If you don't see the "No Versioning" option, make sure the above requirements are met.
User Weropol (https://github.com/Weropol) has found a method to disable versioning on OneDrive
1. Open the settings menu by clicking on the gear symbol at the top of the OneDrive Business page.
2. Click Site settings.
3. Once on the Site settings page, navigate to Site Administration > Site libraries and lists.
4. Click Customize "Documents".
5. Click General Settings > Versioning Settings.
6. Under Document Version History select the option No versioning.  Note: This will disable the creation of new file versions, but will not remove any previous versions.  Your documents are safe.
7. Apply the changes by clicking OK.
8. Use rclone to upload or modify files.  (I also use the --no-update-modtime flag)
9. Restore the versioning settings after using rclone.  (Optional)
Cleanup OneDrive supports rclone cleanup which causes rclone to look through every file under the path supplied and delete all version but the current version.  Because this involves traversing all the files, then querying each file for versions it can be quite slow.  Rclone does --checkers tests in parallel.  The command also supports -i which is a great way to see what it would do.
rclone cleanup -i remote:path/subdir # interactively remove all old version for path/subdir rclone cleanup remote:path/subdir    # unconditionally remove all old version for path/subdir
NB Onedrive personal can't currently delete versions
Troubleshooting Excessive throttling or blocked on SharePoint If you experience excessive throttling or is being blocked on SharePoint then it may help to set the user agent explicitly with a flag like this: --user-agent "ISV|rclone.org|rclone/v1.55.1"
The specific details can be found in the Microsoft document: Avoid getting throttled or blocked in SharePoint Online (https://docs.microsoft.com/en-us/sharepoint/dev/general-development/how- to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to- decorate-your-http-traffic-to-avoid-throttling)
Unexpected file size/hash differences on Sharepoint It is a known (https://github.com/OneDrive/onedrive-api- docs/issues/935#issuecomment-441741631) issue that Sharepoint (not OneDrive or OneDrive for Business) silently modifies uploaded files, mainly Office files (.docx, .xlsx, etc.), causing file size and hash checks to fail.  There are also other situations that will cause OneDrive to report inconsistent file sizes.  To use rclone with such affected files on Sharepoint, you may disable these checks with the following command line arguments:
--ignore-checksum --ignore-size
Alternatively, if you have write access to the OneDrive files, it may be possible to fix this problem for certain files, by attempting the steps below.  Open the web interface for OneDrive (https://onedrive.live.com) and find the affected files (which will be in the error messages/log for rclone).  Simply click on each of these files, causing OneDrive to open them on the web.  This will cause each file to be converted in place to a format that is functionally equivalent but which will no longer trigger the size discrepancy.  Once all problematic files are converted you will no longer need the ignore options above.
Replacing/deleting existing files on Sharepoint gets "item not found" It is a known (https://github.com/OneDrive/onedrive-api-docs/issues/1068) issue that Sharepoint (not OneDrive or OneDrive for Business) may return "item not found" errors when users try to replace or delete uploaded files; this seems to mainly affect Office files (.docx, .xlsx, etc.).  As a workaround, you may use the --backup-dir <BACKUP_DIR> command line argument so rclone moves the files to be replaced/deleted into a given backup directory (instead of directly replacing/deleting them).  For example, to instruct rclone to move the files into the directory rclone-backup-dir on backend mysharepoint, you may use:
--backup-dir mysharepoint:rclone-backup-dir
access_denied (AADSTS65005) Error: access_denied Code: AADSTS65005 Description: Using application 'rclone' is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.
This means that rclone can't use the OneDrive for Business API with your account.  You can't do much about it, maybe write an email to your admins.
However, there are other ways to interact with your OneDrive account. Have a look at the webdav backend: https://rclone.org/webdav/#sharepoint
invalid_grant (AADSTS50076) Error: invalid_grant Code: AADSTS50076 Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '...'.
If you see the error above after enabling multi-factor authentication for your account, you can fix it by refreshing your OAuth refresh token.  To do that, run rclone config, and choose to edit your OneDrive backend. Then, you don't need to actually make any changes until you reach this question: Already have a token - refresh?.  For this question, answer y and go through the process to refresh your token, just like the first time the backend is configured.  After this, rclone should work again for this backend.
Invalid request when making public links On Sharepoint and OneDrive for Business, rclone link may return an "Invalid request" error.  A possible cause is that the organisation admin didn't allow public links to be made for the organisation/sharepoint library.  To fix the permissions as an admin, take a look at the docs: 1 (https://docs.microsoft.com/en-us/sharepoint/turn-external-sharing-on-or- off), 2 (https://support.microsoft.com/en-us/office/set-up-and-manage- access-requests-94b26e0b-2822-49d4-929a-8455698654b3).
OpenDrive Paths are specified as remote:path
n) New remote d) Delete remote q) Quit config e/n/d/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / OpenDrive \ "opendrive" [snip] Storage> opendrive Username username> Password y) Yes type in my own password g) Generate random password y/g> y Enter the password: password: Confirm the password: password: -------------------- [remote] username = password = *** ENCRYPTED *** -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
List directories in top level of your OpenDrive
List all the files in your OpenDrive
To copy a local directory to an OpenDrive directory called backup
Modified time and MD5SUMs OpenDrive allows modification times to be set on objects accurate to 1 second.  These will be used to detect whether objects need syncing or not.
Restricted filename characters Character   Value   Replacement ──────────────────────────────── NUL         0x00         ␀ /           0x2F         ／ "           0x22         ＂ *           0x2A         ＊ :           0x3A         ： <           0x3C         ＜ >           0x3E         ＞ ?           0x3F         ？ \           0x5C         ＼ |           0x7C         ｜
File names can also not begin or end with the following characters. These only get replaced if they are the first or last character in the name:
Character   Value   Replacement ──────────────────────────────── SP          0x20         ␠ HT          0x09         ␉ LF          0x0A         ␊ VT          0x0B         ␋ CR          0x0D         ␍
Standard options Here are the standard options specific to opendrive (OpenDrive).
--opendrive-username Username.
• Env Var: RCLONE_OPENDRIVE_USERNAME
--opendrive-password Password.
• Env Var: RCLONE_OPENDRIVE_PASSWORD
Advanced options Here are the advanced options specific to opendrive (OpenDrive).
--opendrive-encoding This sets the encoding for the backend.
• Env Var: RCLONE_OPENDRIVE_ENCODING
• Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot
--opendrive-chunk-size Files will be uploaded in chunks this size.
Note that these chunks are buffered in memory so increasing them will increase memory use.
• Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE
Limitations Note that OpenDrive is case insensitive so you can't have a file called "Hello.doc" and one called "hello.doc".
There are quite a few characters that can't be in OpenDrive file names. These can't occur on Windows platforms, but on non-Windows platforms they are common.  Rclone will map these names to and from an identical looking unicode equivalent.  For example if a file has a ? in it will be mapped to ？ instead.
rclone about is not supported by the OpenDrive backend.  Backends without this capability cannot determine free space for an rclone mount or use policy mfs (most free space) as a member of an rclone union remote.
QingStor Paths are specified as remote:bucket (or remote: for the lsd command.) You may put subdirectories in too, e.g.  remote:bucket/path/to/dir.
Configuration Here is an example of making an QingStor configuration.  First run
No remotes found - make a new one n) New remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config n/r/c/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / QingStor Object Storage \ "qingstor" [snip] Storage> qingstor Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank. Choose a number from below, or type in your own value 1 / Enter QingStor credentials in the next step \ "false" 2 / Get QingStor credentials from the environment (env vars or IAM) \ "true" env_auth> 1 QingStor Access Key ID - leave blank for anonymous access or runtime credentials. access_key_id> access_key QingStor Secret Access Key (password) - leave blank for anonymous access or runtime credentials. secret_access_key> secret_key Enter an endpoint URL to connection QingStor API. Leave blank will use the default value "https://qingstor.com:443" endpoint> Zone connect to. Default is "pek3a". Choose a number from below, or type in your own value / The Beijing (China) Three Zone 1 | Needs location constraint pek3a. \ "pek3a" / The Shanghai (China) First Zone 2 | Needs location constraint sh1a. \ "sh1a" zone> 1 Number of connection retry. Leave blank will use the default value "3". connection_retries> Remote config -------------------- [remote] env_auth = false access_key_id = access_key secret_access_key = secret_key endpoint = zone = pek3a connection_retries = -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
Multipart uploads rclone supports multipart uploads with QingStor which means that it can upload files bigger than 5 GiB.  Note that files uploaded with multipart upload don't have an MD5SUM.
Note that incomplete multipart uploads older than 24 hours can be removed with rclone cleanup remote:bucket just for one bucket rclone cleanup remote: for all buckets.  QingStor does not ever remove incomplete multipart uploads so it may be necessary to run this from time to time.
Buckets and Zone With QingStor you can list buckets (rclone lsd) using any zone, but you can only access the content of a bucket from the zone it was created in. If you attempt to access a bucket from the wrong zone, you will get an error, incorrect zone, the bucket is not in 'XXX' zone.
Authentication There are two ways to supply rclone with a set of QingStor credentials. In order of precedence:
• Directly in the rclone configuration file (as configured by rclone config)
• set access_key_id and secret_access_key
• Runtime configuration:
• set env_auth to true in the config file
• Exporting the following environment variables before running rclone
• Access Key ID: QS_ACCESS_KEY_ID or QS_ACCESS_KEY
• Secret Access Key: QS_SECRET_ACCESS_KEY or QS_SECRET_KEY
Restricted filename characters The control characters 0x00-0x1F and / are replaced as in the default restricted characters set (https://rclone.org/overview/#restricted- characters).  Note that 0x7F is not replaced.
Standard options Here are the standard options specific to qingstor (QingCloud Object Storage).
--qingstor-env-auth Get QingStor credentials from runtime.
• Env Var: RCLONE_QINGSTOR_ENV_AUTH
• Enter QingStor credentials in the next step.
• Get QingStor credentials from the environment (env vars or IAM).
--qingstor-access-key-id QingStor Access Key ID.
• Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
--qingstor-secret-access-key QingStor Secret Access Key (password).
• Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
--qingstor-endpoint Enter an endpoint URL to connection QingStor API.
Leave blank will use the default value "https://qingstor.com:443".
• Env Var: RCLONE_QINGSTOR_ENDPOINT
--qingstor-zone Zone to connect to.
Default is "pek3a".
• Config: zone
• Env Var: RCLONE_QINGSTOR_ZONE
• "pek3a"
• The Beijing (China) Three Zone.
• Needs location constraint pek3a.
• "sh1a"
• The Shanghai (China) First Zone.
• Needs location constraint sh1a.
• "gd2a"
• The Guangdong (China) Second Zone.
• Needs location constraint gd2a.
Advanced options Here are the advanced options specific to qingstor (QingCloud Object Storage).
--qingstor-connection-retries Number of connection retries.
• Config: connection_retries
• Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
• Default: 3
--qingstor-upload-cutoff Cutoff for switching to chunked upload.
• Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
--qingstor-chunk-size Chunk size to use for uploading.
When uploading files larger than upload_cutoff they will be uploaded as multipart uploads using this chunk size.
Note that "--qingstor-upload-concurrency" chunks of this size are buffered in memory per transfer.
• Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
--qingstor-upload-concurrency Concurrency for multipart uploads.
NB if you set this to > 1 then the checksums of multipart uploads become corrupted (the uploads themselves are not corrupted though).
• Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
--qingstor-encoding This sets the encoding for the backend.
• Env Var: RCLONE_QINGSTOR_ENCODING
• Default: Slash,Ctl,InvalidUtf8
Limitations rclone about is not supported by the qingstor backend.  Backends without this capability cannot determine free space for an rclone mount or use policy mfs (most free space) as a member of an rclone union remote.
Sia Sia (sia.tech (https://sia.tech/)) is a decentralized cloud storage platform based on the blockchain (https://wikipedia.org/wiki/Blockchain) technology.  With rclone you can use it like any other remote filesystem or mount Sia folders locally.  The technology behind it involves a number of new concepts such as Siacoins and Wallet, Blockchain and Consensus, Renting and Hosting, and so on.  If you are new to it, you'd better first familiarize yourself using their excellent support documentation (https://support.sia.tech/).
Introduction Before you can use rclone with Sia, you will need to have a running copy of Sia-UI or siad (the Sia daemon) locally on your computer or on local network (e.g.  a NAS).  Please follow the Get started (https://sia.tech/get-started) guide and install one.
rclone interacts with Sia network by talking to the Sia daemon via HTTP API (https://sia.tech/docs/) which is usually available on port 9980.  By default you will run the daemon locally on the same computer so it's safe to leave the API password blank (the API URL will be http://127.0.0.1:9980 making external access impossible).
However, if you want to access Sia daemon running on another node, for example due to memory constraints or because you want to share single daemon between several rclone and Sia-UI instances, you'll need to make a few more provisions: - Ensure you have Sia daemon installed directly or in a docker container (https://github.com/SiaFoundation/siad/pkgs/container/siad) because Sia-UI does not support this mode natively.  - Run it on externally accessible port, for example provide --api-addr :9980 and --disable-api-security arguments on the daemon command line.  - Enforce API password for the siad daemon via environment variable SIA_API_PASSWORD or text file named apipassword in the daemon directory. - Set rclone backend option api_password taking it from above locations.
Notes: 1.  If your wallet is locked, rclone cannot unlock it automatically.  You should either unlock it in advance by using Sia-UI or via command line siac wallet unlock.  Alternatively you can make siad unlock your wallet automatically upon startup by running it with environment variable SIA_WALLET_PASSWORD.  2.  If siad cannot find the SIA_API_PASSWORD variable or the apipassword file in the SIA_DIR directory, it will generate a random password and store in the text file named apipassword under YOUR_HOME/.sia/ directory on Unix or C:\Users\YOUR_HOME\AppData\Local\Sia\apipassword on Windows.  Remember this when you configure password in rclone.  3.  The only way to use siad without API password is to run it on localhost with command line argument --authorize-api=false, but this is insecure and strongly discouraged.
Configuration Here is an example of how to make a sia remote called mySia.  First, run:
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> mySia Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value ... 29 / Sia Decentralized Cloud \ "sia" ... Storage> sia Sia daemon API URL, like http://sia.daemon.host:9980. Note that siad must run with --disable-api-security to open API port for other hosts (not recommended). Keep default if Sia daemon runs on localhost. Enter a string value. Press Enter for the default ("http://127.0.0.1:9980"). api_url> http://127.0.0.1:9980 Sia Daemon API Password. Can be found in the apipassword file located in HOME/.sia/ or in the daemon directory. y) Yes type in my own password g) Generate random password n) No leave this optional password blank (default) y/g/n> y Enter the password: password: Confirm the password: password: Edit advanced config? y) Yes n) No (default) y/n> n -------------------- [mySia] type = sia api_url = http://127.0.0.1:9980 api_password = *** ENCRYPTED *** -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
Once configured, you can then use rclone like this:
• List directories in top level of your Sia storage
rclone lsd mySia:
• List all the files in your Sia storage
rclone ls mySia:
• Upload a local directory to the Sia directory called backup
rclone copy /home/source mySia:backup
Standard options Here are the standard options specific to sia (Sia Decentralized Cloud).
--sia-api-url Sia daemon API URL, like http://sia.daemon.host:9980.
Note that siad must run with --disable-api-security to open API port for other hosts (not recommended).  Keep default if Sia daemon runs on localhost.
• Config: api_url
• Env Var: RCLONE_SIA_API_URL
• Default: "http://127.0.0.1:9980"
--sia-api-password Sia Daemon API Password.
Can be found in the apipassword file located in HOME/.sia/ or in the daemon directory.
• Config: api_password
• Env Var: RCLONE_SIA_API_PASSWORD
Advanced options Here are the advanced options specific to sia (Sia Decentralized Cloud).
--sia-user-agent Siad User Agent
Sia daemon requires the 'Sia-Agent' user agent by default for security
• Env Var: RCLONE_SIA_USER_AGENT
• Default: "Sia-Agent"
--sia-encoding This sets the encoding for the backend.
• Env Var: RCLONE_SIA_ENCODING
• Default: Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot
Limitations • Modification times not supported
• Checksums not supported
• rclone about not supported
• rclone can work only with Siad or Sia-UI at the moment, the SkyNet daemon is not supported yet.
• Sia does not allow control characters or symbols like question and pound signs in file names.  rclone will transparently encode (https://rclone.org/overview/#encoding) them for you, but you'd better be aware
Swift Swift refers to OpenStack Object Storage (https://docs.openstack.org/swift/latest/).  Commercial implementations of that being:
• Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
• Memset Memstore (https://www.memset.com/cloud/storage/)
• OVH Object Storage (https://www.ovh.co.uk/public-cloud/storage/object- storage/)
• Oracle Cloud Storage (https://cloud.oracle.com/object-storage/buckets)
• IBM Bluemix Cloud ObjectStorage Swift (https://console.bluemix.net/docs/infrastructure/objectstorage- swift/index.html)
Configuration Here is an example of making a swift configuration.  First run