neulab/tldr huggingface dataset:
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" [snip] Storage> swift Get swift credentials from environment variables in standard OpenStack form. Choose a number from below, or type in your own value 1 / Enter swift credentials in the next step \ "false" 2 / Get swift credentials from environment vars. Leave other fields blank if using this. \ "true" env_auth> true User name to log in (OS_USERNAME). user> API key or password (OS_PASSWORD). key> Authentication URL for server (OS_AUTH_URL). Choose a number from below, or type in your own value 1 / Rackspace US \ "https://auth.api.rackspacecloud.com/v1.0" 2 / Rackspace UK \ "https://lon.auth.api.rackspacecloud.com/v1.0" 3 / Rackspace v2 \ "https://identity.api.rackspacecloud.com/v2.0" 4 / Memset Memstore UK \ "https://auth.storage.memset.com/v1.0" 5 / Memset Memstore UK v2 \ "https://auth.storage.memset.com/v2.0" 6 / OVH \ "https://auth.cloud.ovh.net/v3" auth> User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID). user_id> User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME) domain> Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME) tenant> Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID) tenant_id> Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME) tenant_domain> Region name - optional (OS_REGION_NAME) region> Storage URL - optional (OS_STORAGE_URL) storage_url> Auth Token from alternate authentication - optional (OS_AUTH_TOKEN) auth_token> AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION) auth_version> Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) Choose a number from below, or type in your own value 1 / Public (default, choose this if not sure) \ "public" 2 / Internal (use internal service net) \ "internal" 3 / Admin \ "admin" endpoint_type> Remote config -------------------- [test] env_auth = true user = key = auth = user_id = domain = tenant = tenant_id = tenant_domain = region = storage_url = auth_token = auth_version = endpoint_type = -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
Configuration from an OpenStack credentials file An OpenStack credentials file typically looks something something like this (without the comments)
export OS_AUTH_URL=https://a.provider.net/v2.0 export OS_TENANT_ID=ffffffffffffffffffffffffffffffff export OS_TENANT_NAME="1234567890123456" export OS_USERNAME="123abc567xy" echo "Please enter your OpenStack Password: " read -sr OS_PASSWORD_INPUT export OS_PASSWORD=$OS_PASSWORD_INPUT export OS_REGION_NAME="SBG1" if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
The config file needs to look something like this where $OS_USERNAME represents the value of the OS_USERNAME variable - 123abc567xy in the example above.
[remote] type = swift user = $OS_USERNAME key = $OS_PASSWORD auth = $OS_AUTH_URL tenant = $OS_TENANT_NAME
Note that you may (or may not) need to set region too - try without first.
Configuration from the environment If you prefer you can configure rclone to use swift using a standard set of OpenStack environment variables.
When you run through the config, make sure you choose true for env_auth and leave everything else blank.
rclone will then set any empty config parameters from the environment using standard OpenStack environment variables.  There is a list of the variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvironment) in the docs for the swift library.
Using an alternate authentication method If your OpenStack installation uses a non-standard authentication method that might not be yet supported by rclone or the underlying swift library, you can authenticate externally (e.g.  calling manually the openstack commands to get a token).  Then, you just need to pass the two configuration variables auth_token and storage_url.  If they are both provided, the other variables are ignored.  rclone will not try to authenticate but instead assume it is already authenticated and use these two variables to access the OpenStack installation.
Using rclone without a config file You can use rclone with swift without a config file, if desired, like this:
source openstack-credentials-file export RCLONE_CONFIG_MYREMOTE_TYPE=swift export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true rclone lsd myremote:
--update and --use-server-modtime As noted below, the modified time is stored on metadata on the object. It is used by default for all operations that require checking the time a file was last updated.  It allows rclone to treat the remote more like a true filesystem, but it is inefficient because it requires an extra API call to retrieve the metadata.
For many operations, the time the object was last uploaded to the remote is sufficient to determine if it is "dirty".  By using --update along with --use-server-modtime, you can avoid the extra API call and simply upload files whose local modtime is newer than the time it was last uploaded.
Standard options Here are the standard options specific to swift (OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)).
--swift-env-auth Get swift credentials from environment variables in standard OpenStack form.
• Env Var: RCLONE_SWIFT_ENV_AUTH
• Enter swift credentials in the next step.
• Get swift credentials from environment vars.
• Leave other fields blank if using this.
--swift-user User name to log in (OS_USERNAME).
• Env Var: RCLONE_SWIFT_USER
--swift-key API key or password (OS_PASSWORD).
• Env Var: RCLONE_SWIFT_KEY
--swift-auth Authentication URL for server (OS_AUTH_URL).
• Config: auth
• Env Var: RCLONE_SWIFT_AUTH
• "https://auth.api.rackspacecloud.com/v1.0"
• Rackspace US
• "https://lon.auth.api.rackspacecloud.com/v1.0"
• Rackspace UK
• "https://identity.api.rackspacecloud.com/v2.0"
• Rackspace v2
• "https://auth.storage.memset.com/v1.0"
• Memset Memstore UK
• "https://auth.storage.memset.com/v2.0"
• Memset Memstore UK v2
• "https://auth.cloud.ovh.net/v3"
--swift-user-id User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
• Config: user_id
• Env Var: RCLONE_SWIFT_USER_ID
--swift-domain User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
• Config: domain
• Env Var: RCLONE_SWIFT_DOMAIN
--swift-tenant Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME).
• Config: tenant
• Env Var: RCLONE_SWIFT_TENANT
--swift-tenant-id Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID).
• Config: tenant_id
• Env Var: RCLONE_SWIFT_TENANT_ID
--swift-tenant-domain Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME).
• Config: tenant_domain
• Env Var: RCLONE_SWIFT_TENANT_DOMAIN
--swift-region Region name - optional (OS_REGION_NAME).
• Env Var: RCLONE_SWIFT_REGION
--swift-storage-url Storage URL - optional (OS_STORAGE_URL).
• Config: storage_url
• Env Var: RCLONE_SWIFT_STORAGE_URL
--swift-auth-token Auth Token from alternate authentication - optional (OS_AUTH_TOKEN).
• Config: auth_token
• Env Var: RCLONE_SWIFT_AUTH_TOKEN
--swift-application-credential-id Application Credential ID (OS_APPLICATION_CREDENTIAL_ID).
• Config: application_credential_id
• Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
--swift-application-credential-name Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME).
• Config: application_credential_name
• Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
--swift-application-credential-secret Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET).
• Config: application_credential_secret
• Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
--swift-auth-version AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION).
• Config: auth_version
• Env Var: RCLONE_SWIFT_AUTH_VERSION
--swift-endpoint-type Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE).
• Config: endpoint_type
• Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
• Default: "public"
• "public"
• Public (default, choose this if not sure)
• "internal"
• Internal (use internal service net)
• "admin"
• Admin
--swift-storage-policy The storage policy to use when creating a new container.
This applies the specified storage policy when creating a new container. The policy cannot be changed afterwards.  The allowed configuration values and their meaning depend on your Swift storage provider.
• Config: storage_policy
• Env Var: RCLONE_SWIFT_STORAGE_POLICY
• "pcs"
• OVH Public Cloud Storage
• "pca"
• OVH Public Cloud Archive
Advanced options Here are the advanced options specific to swift (OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)).
--swift-leave-parts-on-error If true avoid calling abort upload on a failure.
• Env Var: RCLONE_SWIFT_LEAVE_PARTS_ON_ERROR
--swift-chunk-size Above this size files will be chunked into a _segments container.
• Env Var: RCLONE_SWIFT_CHUNK_SIZE
--swift-no-chunk Don't chunk files during streaming upload.
• Env Var: RCLONE_SWIFT_NO_CHUNK
--swift-encoding This sets the encoding for the backend.
• Env Var: RCLONE_SWIFT_ENCODING
Limitations The Swift API doesn't return a correct MD5SUM for segmented files (Dynamic or Static Large Objects) so rclone won't check or use the MD5SUM for these.
Troubleshooting Rclone gives Failed to create file system for "remote:": Bad Request Due to an oddity of the underlying swift library, it gives a "Bad Request" error rather than a more sensible error when the authentication fails for Swift.
So this most likely means your username / password is wrong.  You can investigate further with the --dump-bodies flag.
This may also be caused by specifying the region when you shouldn't have (e.g.  OVH).
Rclone gives Failed to create file system: Response didn't have storage url and auth token This is most likely caused by forgetting to specify your tenant when setting up a swift remote.
OVH Cloud Archive To use rclone with OVH cloud archive, first use rclone config to set up a swift backend with OVH, choosing pca as the storage_policy.
Uploading Objects Uploading objects to OVH cloud archive is no different to object storage, you just simply run the command you like (move, copy or sync) to upload the objects.  Once uploaded the objects will show in a "Frozen" state within the OVH control panel.
Retrieving Objects To retrieve objects use rclone copy as normal.  If the objects are in a frozen state then rclone will ask for them all to be unfrozen and it will wait at the end of the output with a message like the following:
2019/03/23 13:06:33 NOTICE: Received retry after error - sleeping until 2019-03-23T13:16:33.481657164+01:00 (9m59.99985121s)
Rclone will wait for the time specified then retry the copy.
pCloud Paths are specified as remote:path
Configuration The initial setup for pCloud involves getting a token from pCloud which you need to do in your browser.  rclone config walks you through it.
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / Pcloud \ "pcloud" [snip] Storage> pcloud Pcloud App Client Id - leave blank normally. client_id> Pcloud App Client Secret - leave blank normally. client_secret> Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes n) No y/n> y If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code -------------------- [remote] client_id = client_secret = token = {"access_token":"XXX","token_type":"bearer","expiry":"0001-01-01T00:00:00Z"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
Note that rclone runs a webserver on your local machine to collect the token as returned from pCloud.  This only runs from the moment it opens your browser to the moment you get back the verification code.  This is on http://127.0.0.1:53682/ and this it may require you to unblock it temporarily if you are running a host firewall.
List directories in top level of your pCloud
List all the files in your pCloud
To copy a local directory to an pCloud directory called backup
Modified time and hashes pCloud allows modification times to be set on objects accurate to 1 second.  These will be used to detect whether objects need syncing or not.  In order to set a Modification time pCloud requires the object be re-uploaded.
pCloud supports MD5 and SHA1 hashes in the US region, and SHA1 and SHA256 hashes in the EU region, so you can use the --checksum flag.
Deleting files Deleted files will be moved to the trash.  Your subscription level will determine how long items stay in the trash.  rclone cleanup can be used to empty the trash.
Root folder ID You can set the root_folder_id for rclone.  This is the directory (identified by its Folder ID) that rclone considers to be the root of your pCloud drive.
In order to do this you will have to find the Folder ID of the directory you wish rclone to display.  This will be the folder field of the URL when you open the relevant folder in the pCloud web interface.
So if the folder you want rclone to use has a URL which looks like https://my.pcloud.com/#page=filemanager&folder=5xxxxxxxx8&tpl=foldergrid in the browser, then you use 5xxxxxxxx8 as the root_folder_id in the config.
Standard options Here are the standard options specific to pcloud (Pcloud).
--pcloud-client-id OAuth Client Id.
• Env Var: RCLONE_PCLOUD_CLIENT_ID
--pcloud-client-secret OAuth Client Secret.
• Env Var: RCLONE_PCLOUD_CLIENT_SECRET
Advanced options Here are the advanced options specific to pcloud (Pcloud).
--pcloud-token OAuth Access Token as a JSON blob.
• Env Var: RCLONE_PCLOUD_TOKEN
--pcloud-auth-url Auth server URL.
• Env Var: RCLONE_PCLOUD_AUTH_URL
--pcloud-token-url Token server url.
• Env Var: RCLONE_PCLOUD_TOKEN_URL
--pcloud-encoding This sets the encoding for the backend.
• Env Var: RCLONE_PCLOUD_ENCODING
--pcloud-root-folder-id Fill in for rclone to use a non root folder as its starting point.
• Env Var: RCLONE_PCLOUD_ROOT_FOLDER_ID
• Default: "d0"
--pcloud-hostname Hostname to connect to.
This is normally set when rclone initially does the oauth connection, however you will need to set it by hand if you are using remote config with rclone authorize.
• Config: hostname
• Env Var: RCLONE_PCLOUD_HOSTNAME
• Default: "api.pcloud.com"
• "api.pcloud.com"
• Original/US region
• "eapi.pcloud.com"
• EU region
premiumize.me Paths are specified as remote:path
Configuration The initial setup for premiumize.me (https://premiumize.me/) involves getting a token from premiumize.me which you need to do in your browser. rclone config walks you through it.
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / premiumize.me \ "premiumizeme" [snip] Storage> premiumizeme ** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ **
Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes n) No y/n> y If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code -------------------- [remote] type = premiumizeme token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2029-08-07T18:44:15.548915378+01:00"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d>
Note that rclone runs a webserver on your local machine to collect the token as returned from premiumize.me.  This only runs from the moment it opens your browser to the moment you get back the verification code. This is on http://127.0.0.1:53682/ and this it may require you to unblock it temporarily if you are running a host firewall.
List directories in top level of your premiumize.me
List all the files in your premiumize.me
To copy a local directory to an premiumize.me directory called backup
Modified time and hashes premiumize.me does not support modification times or hashes, therefore syncing will default to --size-only checking.  Note that using --update will work.
Character   Value   Replacement ──────────────────────────────── \           0x5C         ＼ "           0x22         ＂
Standard options Here are the standard options specific to premiumizeme (premiumize.me).
--premiumizeme-api-key API Key.
This is not normally used - use oauth instead.
• Env Var: RCLONE_PREMIUMIZEME_API_KEY
Advanced options Here are the advanced options specific to premiumizeme (premiumize.me).
--premiumizeme-encoding This sets the encoding for the backend.
• Env Var: RCLONE_PREMIUMIZEME_ENCODING
• Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot
Limitations Note that premiumize.me is case insensitive so you can't have a file called "Hello.doc" and one called "hello.doc".
premiumize.me file names can't have the \ or " characters in.  rclone maps these to and from an identical looking unicode equivalents ＼ and ＂
premiumize.me only supports filenames up to 255 characters in length.
put.io Paths are specified as remote:path
put.io paths may be as deep as required, e.g. remote:directory/subdirectory.
Configuration The initial setup for put.io involves getting a token from put.io which you need to do in your browser.  rclone config walks you through it.
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> putio Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Put.io \ "putio" [snip] Storage> putio ** See help for putio backend at: https://rclone.org/putio/ **
Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes n) No y/n> y If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code -------------------- [putio] type = putio token = {"access_token":"XXXXXXXX","expiry":"0001-01-01T00:00:00Z"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y Current remotes:
Name                 Type ====                 ==== putio                putio
List directories in top level of your put.io
List all the files in your put.io
To copy a local directory to a put.io directory called backup
Advanced options Here are the advanced options specific to putio (Put.io).
--putio-encoding This sets the encoding for the backend.
• Env Var: RCLONE_PUTIO_ENCODING
Seafile This is a backend for the Seafile (https://www.seafile.com/) storage service: - It works with both the free community edition or the professional edition.  - Seafile versions 6.x and 7.x are all supported. - Encrypted libraries are also supported.  - It supports 2FA enabled users
Configuration There are two distinct modes you can setup your remote: - you point your remote to the root of the server, meaning you don't specify a library during the configuration: Paths are specified as remote:library.  You may put subdirectories in too, e.g.  remote:library/path/to/dir.  - you point your remote to a specific library during the configuration: Paths are specified as remote:path/to/dir.  This is the recommended mode when using encrypted libraries.  (This mode is possibly slightly faster than the root mode)
Configuration in root mode Here is an example of making a seafile configuration for a user with no two-factor authentication.  First run
This will guide you through an interactive setup process.  To authenticate you will need the URL of your server, your email (or username) and your password.
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> seafile Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Seafile \ "seafile" [snip] Storage> seafile ** See help for seafile backend at: https://rclone.org/seafile/ **
URL of seafile host to connect to Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Connect to cloud.seafile.com \ "https://cloud.seafile.com/" url> http://my.seafile.server/ User name (usually email address) Enter a string value. Press Enter for the default (""). user> me@example.com Password y) Yes type in my own password g) Generate random password n) No leave this optional password blank (default) y/g> y Enter the password: password: Confirm the password: password: Two-factor authentication ('true' if the account has 2FA enabled) Enter a boolean value (true or false). Press Enter for the default ("false"). 2fa> false Name of the library. Leave blank to access all non-encrypted libraries. Enter a string value. Press Enter for the default (""). library> Library password (for encrypted libraries only). Leave blank if you pass it through the command line. y) Yes type in my own password g) Generate random password n) No leave this optional password blank (default) y/g/n> n Edit advanced config? (y/n) y) Yes n) No (default) y/n> n Remote config Two-factor authentication is not enabled on this account. -------------------- [seafile] type = seafile url = http://my.seafile.server/ user = me@example.com pass = *** ENCRYPTED *** 2fa = false -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
This remote is called seafile.  It's pointing to the root of your seafile server and can now be used like this:
See all libraries
rclone lsd seafile:
Create a new library
rclone mkdir seafile:library
List the contents of a library
rclone ls seafile:library
Sync /home/local/directory to the remote library, deleting any excess files in the library.
rclone sync -i /home/local/directory seafile:library
Configuration in library mode Here's an example of a configuration in library mode with a user that has the two-factor authentication enabled.  Your 2FA code will be asked at the end of the configuration, and will attempt to authenticate you:
URL of seafile host to connect to Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Connect to cloud.seafile.com \ "https://cloud.seafile.com/" url> http://my.seafile.server/ User name (usually email address) Enter a string value. Press Enter for the default (""). user> me@example.com Password y) Yes type in my own password g) Generate random password n) No leave this optional password blank (default) y/g> y Enter the password: password: Confirm the password: password: Two-factor authentication ('true' if the account has 2FA enabled) Enter a boolean value (true or false). Press Enter for the default ("false"). 2fa> true Name of the library. Leave blank to access all non-encrypted libraries. Enter a string value. Press Enter for the default (""). library> My Library Library password (for encrypted libraries only). Leave blank if you pass it through the command line. y) Yes type in my own password g) Generate random password n) No leave this optional password blank (default) y/g/n> n Edit advanced config? (y/n) y) Yes n) No (default) y/n> n Remote config Two-factor authentication: please enter your 2FA code 2fa code> 123456 Authenticating... Success! -------------------- [seafile] type = seafile url = http://my.seafile.server/ user = me@example.com pass = 2fa = true library = My Library -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
You'll notice your password is blank in the configuration.  It's because we only need the password to authenticate you once.
You specified My Library during the configuration.  The root of the remote is pointing at the root of the library My Library:
See all files in the library:
Create a new directory inside the library
rclone mkdir seafile:directory
rclone ls seafile:directory
rclone sync -i /home/local/directory seafile:
--fast-list Seafile version 7+ supports --fast-list which allows you to use fewer transactions in exchange for more memory.  See the rclone docs (https://rclone.org/docs/#fast-list) for more details.  Please note this is not supported on seafile server version 6.x
Character   Value   Replacement ──────────────────────────────── /           0x2F         ／ "           0x22         ＂ \           0x5C         ＼
Seafile and rclone link Rclone supports generating share links for non-encrypted libraries only. They can either be for a file or a directory:
rclone link seafile:seafile-tutorial.doc http://my.seafile.server/f/fdcd8a2f93f84b8b90f4/
rclone link seafile:dir http://my.seafile.server/d/9ea2455f6f55478bbb0d/
Please note a share link is unique for each file or directory.  If you run a link command on a file/dir that has already been shared, you will get the exact same link.
Compatibility It has been actively tested using the seafile docker image (https://github.com/haiwen/seafile-docker) of these versions: - 6.3.4 community edition - 7.0.5 community edition - 7.1.3 community edition
Versions below 6.0 are not supported.  Versions between 6.0 and 6.3 haven't been tested and might not work properly.
Standard options Here are the standard options specific to seafile (seafile).
--seafile-url URL of seafile host to connect to.
• Env Var: RCLONE_SEAFILE_URL
• "https://cloud.seafile.com/"
• Connect to cloud.seafile.com.
--seafile-user User name (usually email address).
• Env Var: RCLONE_SEAFILE_USER
--seafile-pass Password.
• Env Var: RCLONE_SEAFILE_PASS
--seafile-2fa Two-factor authentication ('true' if the account has 2FA enabled).
• Config: 2fa
• Env Var: RCLONE_SEAFILE_2FA
--seafile-library Name of the library.
Leave blank to access all non-encrypted libraries.
• Config: library
• Env Var: RCLONE_SEAFILE_LIBRARY
--seafile-library-key Library password (for encrypted libraries only).
Leave blank if you pass it through the command line.
• Config: library_key
• Env Var: RCLONE_SEAFILE_LIBRARY_KEY
--seafile-auth-token Authentication token.
• Env Var: RCLONE_SEAFILE_AUTH_TOKEN
Advanced options Here are the advanced options specific to seafile (seafile).
--seafile-create-library Should rclone create a library if it doesn't exist.
• Config: create_library
• Env Var: RCLONE_SEAFILE_CREATE_LIBRARY
--seafile-encoding This sets the encoding for the backend.
• Env Var: RCLONE_SEAFILE_ENCODING
• Default: Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8
SFTP is the Secure (or SSH) File Transfer Protocol (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
The SFTP backend can be used with a number of different providers:
SFTP runs over SSH v2 and is installed as standard with most modern SSH installations.
Paths are specified as remote:path.  If the path does not begin with a / it is relative to the home directory of the user.  An empty path remote: refers to the user's home directory.  For example, rclone lsd remote: would list the home directory of the user cofigured in the rclone remote config (i.e /home/sftpuser).  However, rclone lsd remote:/ would list the root directory for remote machine (i.e.  /)
Note that some SFTP servers will need the leading / - Synology is a good example of this.  rsync.net, on the other hand, requires users to OMIT the leading /.
Configuration Here is an example of making an SFTP configuration.  First run
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / SSH/SFTP Connection \ "sftp" [snip] Storage> sftp SSH host to connect to Choose a number from below, or type in your own value 1 / Connect to example.com \ "example.com" host> example.com SSH username, leave blank for current username, $USER user> sftpuser SSH port, leave blank to use default (22) port> SSH password, leave blank to use ssh-agent. y) Yes type in my own password g) Generate random password n) No leave this optional password blank y/g/n> n Path to unencrypted PEM-encoded private key file, leave blank to use ssh-agent. key_file> Remote config -------------------- [remote] host = example.com user = sftpuser port = pass = key_file = -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
This remote is called remote and can now be used like this:
See all directories in the home directory
See all directories in the root directory
rclone lsd remote:/
Mount the remote path /srv/www-data/ to the local path /mnt/www-data
rclone mount remote:/srv/www-data/ /mnt/www-data
SSH Authentication The SFTP remote supports three authentication methods:
• Password
• Key file, including certificate signed keys
• ssh-agent
Key files should be PEM-encoded private key files.  For instance /home/$USER/.ssh/id_rsa.  Only unencrypted OpenSSH or PEM encrypted files are supported.
The key file can be specified in either an external file (key_file) or contained within the rclone config file (key_pem).  If using key_pem in the config file, the entry should be on a single line with new line ('' or '') separating lines.  i.e.
key_pem = -----BEGIN RSA PRIVATE KEY-----\nMaMbaIXtE\n0gAMbMbaSsd\nMbaass\n-----END RSA PRIVATE KEY-----
This will generate it correctly for key_pem for use in the config:
awk '{printf "%s\\n", $0}' < ~/.ssh/id_rsa
If you don't specify pass, key_file, or key_pem or ask_password then rclone will attempt to contact an ssh-agent.  You can also specify key_use_agent to force the usage of an ssh-agent.  In this case key_file or key_pem can also be specified to force the usage of a specific key in the ssh-agent.
Using an ssh-agent is the only way to load encrypted OpenSSH keys at the moment.
If you set the ask_password option, rclone will prompt for a password when needed and no password has been configured.
Certificate-signed keys With traditional key-based authentication, you configure your private key only, and the public key built into it will be used during the authentication process.
If you have a certificate you may use it to sign your public key, creating a separate SSH user certificate that should be used instead of the plain public key extracted from the private key.  Then you must provide the path to the user certificate public key file in pubkey_file.
Note: This is not the traditional public key paired with your private key, typically saved as /home/$USER/.ssh/id_rsa.pub.  Setting this path in pubkey_file will not work.
[remote] type = sftp host = example.com user = sftpuser key_file = ~/id_rsa pubkey_file = ~/id_rsa-cert.pub
If you concatenate a cert with a private key then you can specify the merged file in both places.
Note: the cert must come first in the file.  e.g.
cat id_rsa-cert.pub id_rsa > merged_key
Host key validation By default rclone will not check the server's host key for validation. This can allow an attacker to replace a server with their own and if you use password authentication then this can lead to that password being exposed.
Host key matching, using standard known_hosts files can be turned on by enabling the known_hosts_file option.  This can point to the file maintained by OpenSSH or can point to a unique file.
[remote] type = sftp host = example.com user = sftpuser pass = known_hosts_file = ~/.ssh/known_hosts
There are some limitations:
• rclone will not manage this file for you.  If the key is missing or wrong then the connection will be refused.
• If the server is set up for a certificate host key then the entry in the known_hosts file must be the @cert-authority entry for the CA
• Unlike OpenSSH, the libraries used by rclone do not permit (at time of writing) multiple host keys to be listed for a server.  Only the first entry is used.
If the host key provided by the server does not match the one in the file (or is missing) then the connection will be aborted and an error returned such as
NewFs: couldn't connect SSH: ssh: handshake failed: knownhosts: key mismatch
NewFs: couldn't connect SSH: ssh: handshake failed: knownhosts: key is unknown
If you see an error such as
NewFs: couldn't connect SSH: ssh: handshake failed: ssh: no authorities for hostname: example.com:22
then it is likely the server has presented a CA signed host certificate and you will need to add the appropriate @cert-authority entry.
The known_hosts_file setting can be set during rclone config as an advanced option.
ssh-agent on macOS Note that there seem to be various problems with using an ssh-agent on macOS due to recent changes in the OS.  The most effective work-around seems to be to start an ssh-agent in each session, e.g.
eval `ssh-agent -s` && ssh-add -A
And then at the end of the session
eval `ssh-agent -k`
These commands can be used in scripts of course.
Modified time Modified times are stored on the server to 1 second precision.
Modified times are used in syncing and are fully supported.
Some SFTP servers disable setting/modifying the file modification time after upload (for example, certain configurations of ProFTPd with mod_sftp).  If you are using one of these servers, you can set the option set_modtime = false in your RClone backend configuration to disable this behaviour.
Standard options Here are the standard options specific to sftp (SSH/SFTP Connection).
--sftp-host SSH host to connect to.
E.g.  "example.com".
• Env Var: RCLONE_SFTP_HOST
--sftp-user SSH username, leave blank for current username, $USER.
• Env Var: RCLONE_SFTP_USER
--sftp-port SSH port, leave blank to use default (22).
• Env Var: RCLONE_SFTP_PORT
--sftp-pass SSH password, leave blank to use ssh-agent.
• Env Var: RCLONE_SFTP_PASS
--sftp-key-pem Raw PEM-encoded private key.
If specified, will override key_file parameter.
• Config: key_pem
• Env Var: RCLONE_SFTP_KEY_PEM
--sftp-key-file Path to PEM-encoded private key file.
Leave blank or set key-use-agent to use ssh-agent.
• Config: key_file
• Env Var: RCLONE_SFTP_KEY_FILE
--sftp-key-file-pass The passphrase to decrypt the PEM-encoded private key file.
Only PEM encrypted key files (old OpenSSH format) are supported. Encrypted keys in the new OpenSSH format can't be used.
• Config: key_file_pass
• Env Var: RCLONE_SFTP_KEY_FILE_PASS
--sftp-pubkey-file Optional path to public key file.
Set this if you have a signed certificate you want to use for authentication.
• Config: pubkey_file
• Env Var: RCLONE_SFTP_PUBKEY_FILE
--sftp-key-use-agent When set forces the usage of the ssh-agent.
When key-file is also set, the ".pub" file of the specified key-file is read and only the associated key is requested from the ssh-agent.  This allows to avoid Too many authentication failures for *username* errors when the ssh-agent contains many keys.
• Config: key_use_agent
• Env Var: RCLONE_SFTP_KEY_USE_AGENT
--sftp-use-insecure-cipher Enable the use of insecure ciphers and key exchange methods.
This enables the use of the following insecure ciphers and key exchange methods:
• aes128-cbc
• aes192-cbc
• aes256-cbc
• 3des-cbc
• diffie-hellman-group-exchange-sha256
• diffie-hellman-group-exchange-sha1
Those algorithms are insecure and may allow plaintext data to be recovered by an attacker.
• Config: use_insecure_cipher
• Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
• Use default Cipher list.
• Enables the use of the aes128-cbc cipher and diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key exchange.
--sftp-disable-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available.
Leave blank or set to false to enable hashing (recommended), set to true to disable hashing.
• Config: disable_hashcheck
• Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
Advanced options Here are the advanced options specific to sftp (SSH/SFTP Connection).
--sftp-known-hosts-file Optional path to known_hosts file.
Set this value to enable server host key validation.
• Config: known_hosts_file
• Env Var: RCLONE_SFTP_KNOWN_HOSTS_FILE
• "~/.ssh/known_hosts"
• Use OpenSSH's known_hosts file.
--sftp-ask-password Allow asking for SFTP password when needed.
If this is set and no password is supplied then rclone will: - ask for a password - not contact the ssh agent
• Config: ask_password
• Env Var: RCLONE_SFTP_ASK_PASSWORD
--sftp-path-override Override path used by SSH connection.
This allows checksum calculation when SFTP and SSH paths are different. This issue affects among others Synology NAS boxes.
Shared folders can be found in directories representing volumes
rclone sync /home/local/directory remote:/directory --ssh-path-override /volume2/directory
Home directory can be found in a shared folder called "home"
rclone sync /home/local/directory remote:/home/directory --ssh-path-override /volume1/homes/USER/directory
• Config: path_override
• Env Var: RCLONE_SFTP_PATH_OVERRIDE
--sftp-set-modtime Set the modified time on the remote if set.
• Config: set_modtime
• Env Var: RCLONE_SFTP_SET_MODTIME
--sftp-md5sum-command The command used to read md5 hashes.
Leave blank for autodetect.
• Config: md5sum_command
• Env Var: RCLONE_SFTP_MD5SUM_COMMAND
--sftp-sha1sum-command The command used to read sha1 hashes.
• Config: sha1sum_command
• Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
--sftp-skip-links Set to skip any symlinks and any other non regular files.
• Config: skip_links
• Env Var: RCLONE_SFTP_SKIP_LINKS
--sftp-subsystem Specifies the SSH2 subsystem on the remote host.
• Config: subsystem
• Env Var: RCLONE_SFTP_SUBSYSTEM
• Default: "sftp"
--sftp-server-command Specifies the path or command to run a sftp server on the remote host.
The subsystem option is ignored when server_command is defined.
• Config: server_command
• Env Var: RCLONE_SFTP_SERVER_COMMAND
--sftp-use-fstat If set use fstat instead of stat.
Some servers limit the amount of open files and calling Stat after opening the file will throw an error from the server.  Setting this flag will call Fstat instead of Stat which is called on an already open file handle.
It has been found that this helps with IBM Sterling SFTP servers which have "extractability" level set to 1 which means only 1 file can be opened at any given time.
• Config: use_fstat
• Env Var: RCLONE_SFTP_USE_FSTAT
--sftp-disable-concurrent-reads If set don't use concurrent reads.
Normally concurrent reads are safe to use and not using them will degrade performance, so this option is disabled by default.
Some servers limit the amount number of times a file can be downloaded. Using concurrent reads can trigger this limit, so if you have a server which returns
Failed to copy: file does not exist
Then you may need to enable this flag.
If concurrent reads are disabled, the use_fstat option is ignored.
• Config: disable_concurrent_reads
• Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_READS
--sftp-disable-concurrent-writes If set don't use concurrent writes.
Normally rclone uses concurrent writes to upload files.  This improves the performance greatly, especially for distant servers.
This option disables concurrent writes should that be necessary.
• Config: disable_concurrent_writes
• Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_WRITES
--sftp-idle-timeout Max time before closing idle connections.
• Env Var: RCLONE_SFTP_IDLE_TIMEOUT
Limitations SFTP supports checksums if the same login has shell access and md5sum or sha1sum as well as echo are in the remote's PATH.  This remote checksumming (file hashing) is recommended and enabled by default. Disabling the checksumming may be required if you are connecting to SFTP servers which are not under your control, and to which the execution of remote commands is prohibited.  Set the configuration option disable_hashcheck to true to disable checksumming.
SFTP also supports about if the same login has shell access and df are in the remote's PATH.  about will return the total space, free space, and used space on the remote for the disk of the specified path on the remote or, if not set, the disk of the root on the remote.  about will fail if it does not have shell access or if df is not in the remote's PATH.
Note that some SFTP servers (e.g.  Synology) the paths are different for SSH and SFTP so the hashes can't be calculated properly.  For them using disable_hashcheck is a good idea.
The only ssh agent supported under Windows is Putty's pageant.
The Go SSH library disables the use of the aes128-cbc cipher by default, due to security concerns.  This can be re-enabled on a per-connection basis by setting the use_insecure_cipher setting in the configuration file to true.  Further details on the insecurity of this cipher can be found in this paper (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
SFTP isn't supported under plan9 until this issue (https://github.com/pkg/sftp/issues/156) is fixed.
Note that since SFTP isn't HTTP based the following flags don't work with it: --dump-headers, --dump-bodies, --dump-auth
Note that --timeout isn't supported (but --contimeout is).
C14 C14 is supported through the SFTP backend.
See C14's documentation (https://www.online.net/en/storage/c14-cold- storage)
rsync.net rsync.net is supported through the SFTP backend.
See rsync.net's documentation of rclone examples (https://www.rsync.net/products/rclone.html).
SugarSync SugarSync (https://sugarsync.com) is a cloud service that enables active synchronization of files across computers and other devices for file backup, access, syncing, and sharing.
Configuration The initial setup for SugarSync involves getting a token from SugarSync which you can do with rclone.  rclone config walks you through it.
No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Sugarsync \ "sugarsync" [snip] Storage> sugarsync ** See help for sugarsync backend at: https://rclone.org/sugarsync/ **
Sugarsync App ID. Leave blank to use rclone's. Enter a string value. Press Enter for the default (""). app_id> Sugarsync Access Key ID. Leave blank to use rclone's. Enter a string value. Press Enter for the default (""). access_key_id> Sugarsync Private Access Key Leave blank to use rclone's. Enter a string value. Press Enter for the default (""). private_access_key> Permanently delete files if true otherwise put them in the deleted files. Enter a boolean value (true or false). Press Enter for the default ("false"). hard_delete> Edit advanced config? (y/n) y) Yes n) No (default) y/n> n Remote config Username (email address)> nick@craig-wood.com Your Sugarsync password is only required during setup and will not be stored. password: -------------------- [remote] type = sugarsync refresh_token = https://api.sugarsync.com/app-authorization/XXXXXXXXXXXXXXXXXX -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
Note that the config asks for your email and password but doesn't store them, it only uses them to get the initial token.
List directories (sync folders) in top level of your SugarSync
List all the files in your SugarSync folder "Test"
rclone ls remote:Test
To copy a local directory to an SugarSync folder called backup
NB you can't create files in the top level folder you have to create a folder, which rclone will create as a "Sync Folder" with SugarSync.
Modified time and hashes SugarSync does not support modification times or hashes, therefore syncing will default to --size-only checking.  Note that using --update will work as rclone can read the time files were uploaded.
Restricted filename characters SugarSync replaces the default restricted characters set (https://rclone.org/overview/#restricted-characters) except for DEL.
Deleting files Deleted files will be moved to the "Deleted items" folder by default.
However you can supply the flag --sugarsync-hard-delete or set the config parameter hard_delete = true if you would like files to be deleted straight away.
Standard options Here are the standard options specific to sugarsync (Sugarsync).
--sugarsync-app-id Sugarsync App ID.
Leave blank to use rclone's.
• Config: app_id
• Env Var: RCLONE_SUGARSYNC_APP_ID
--sugarsync-access-key-id Sugarsync Access Key ID.
• Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID
--sugarsync-private-access-key Sugarsync Private Access Key.
• Config: private_access_key
• Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY
--sugarsync-hard-delete Permanently delete files if true otherwise put them in the deleted files.
• Env Var: RCLONE_SUGARSYNC_HARD_DELETE
Advanced options Here are the advanced options specific to sugarsync (Sugarsync).
--sugarsync-refresh-token Sugarsync refresh token.
Leave blank normally, will be auto configured by rclone.
• Config: refresh_token
• Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN
--sugarsync-authorization Sugarsync authorization.
• Config: authorization
• Env Var: RCLONE_SUGARSYNC_AUTHORIZATION
--sugarsync-authorization-expiry Sugarsync authorization expiry.
• Config: authorization_expiry
• Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY
--sugarsync-user Sugarsync user.
• Env Var: RCLONE_SUGARSYNC_USER
--sugarsync-root-id Sugarsync root id.
• Config: root_id
• Env Var: RCLONE_SUGARSYNC_ROOT_ID
--sugarsync-deleted-id Sugarsync deleted folder id.
• Config: deleted_id
• Env Var: RCLONE_SUGARSYNC_DELETED_ID
--sugarsync-encoding This sets the encoding for the backend.
• Env Var: RCLONE_SUGARSYNC_ENCODING
• Default: Slash,Ctl,InvalidUtf8,Dot
Limitations rclone about is not supported by the SugarSync backend.  Backends without this capability cannot determine free space for an rclone mount or use policy mfs (most free space) as a member of an rclone union remote.
Tardigrade Tardigrade (https://tardigrade.io) is an encrypted, secure, and cost-effective object storage service that enables you to store, back up, and archive large amounts of data in a decentralized manner.
Configuration To make a new Tardigrade configuration you need one of the following: * Access Grant that someone else shared with you.  * API Key (https://documentation.tardigrade.io/getting-started/uploading-your- first-object/create-an-api-key) of a Tardigrade project you are a member of.
Setup with access grant No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Tardigrade Decentralized Cloud Storage \ "tardigrade" [snip] Storage> tardigrade ** See help for tardigrade backend at: https://rclone.org/tardigrade/ **
Choose an authentication method. Enter a string value. Press Enter for the default ("existing"). Choose a number from below, or type in your own value 1 / Use an existing access grant. \ "existing" 2 / Create a new access grant from satellite address, API key, and passphrase. \ "new" provider> existing Access Grant. Enter a string value. Press Enter for the default (""). access_grant> your-access-grant-received-by-someone-else Remote config -------------------- [remote] type = tardigrade access_grant = your-access-grant-received-by-someone-else -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
Setup with API key and passphrase No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> remote Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value [snip] XX / Tardigrade Decentralized Cloud Storage \ "tardigrade" [snip] Storage> tardigrade ** See help for tardigrade backend at: https://rclone.org/tardigrade/ **