neulab/tldr huggingface dataset:
--tpslimit float Limit transactions per second to this number.  Default is 0 which is used to mean unlimited transactions per second.
A transaction is roughly defined as an API call; its exact meaning will depend on the backend.  For HTTP based backends it is an HTTP PUT/GET/POST/etc and its response.  For FTP/SFTP it is a round trip transaction over TCP.
For example to limit rclone to 10 transactions per second use --tpslimit 10, or to 1 transaction every 2 seconds use --tpslimit 0.5.
Use this when the number of transactions per second from rclone is causing a problem with the cloud storage provider (e.g.  getting you banned or rate limited).
This can be very useful for rclone mount to control the behaviour of applications using it.
This limit applies to all HTTP based backends and to the FTP and SFTP backends.  It does not apply to the local backend or the Tardigrade backend.
See also --tpslimit-burst.
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1).
Normally --tpslimit will do exactly the number of transaction per second specified.  However if you supply --tps-burst then rclone can save up some transactions from when it was idle giving a burst of up to the parameter supplied.
For example if you provide --tpslimit-burst 10 then if rclone has been idle for more than 10*--tpslimit then it can do 10 transactions very quickly before they are limited again.
This may be used to increase performance of --tpslimit without changing the long term average number of transactions per second.
--track-renames By default, rclone doesn't keep track of renamed files, so if you rename a file locally then sync it to a remote, rclone will delete the old file on the remote and upload a new copy.
If you use this flag, and the remote supports server-side copy or server-side move, and the source and destination have a compatible hash, then this will track renames during sync operations and perform renaming server-side.
Files will be matched by size and hash - if both match then a rename will be considered.
If the destination does not support server-side copy or move, rclone will fall back to the default behaviour and log an error level message to the console.
Encrypted destinations are not currently supported by --track-renames if --track-renames-strategy includes hash.
Note that --track-renames is incompatible with --no-traverse and that it uses extra memory to keep track of all the rename candidates.
Note also that --track-renames is incompatible with --delete-before and will select --delete-after instead of --delete-during.
--track-renames-strategy (hash,modtime,leaf,size) This option changes the matching criteria for --track-renames.
The matching is controlled by a comma separated selection of these tokens:
• modtime - the modification time of the file - not supported on all backends
• hash - the hash of the file contents - not supported on all backends
• leaf - the name of the file not including its directory name
• size - the size of the file (this is always enabled)
So using --track-renames-strategy modtime,leaf would match files based on modification time, the leaf of the file name and the size only.
Using --track-renames-strategy modtime or leaf can enable --track-renames support for encrypted destinations.
If nothing is specified, the default option is matching by hashes.
Note that the hash strategy is not supported with encrypted destinations.
--delete-(before,during,after) This option allows you to specify when files on your destination are deleted when you sync folders.
Specifying the value --delete-before will delete all files present on the destination, but not on the source before starting the transfer of any new or updated files.  This uses two passes through the file systems, one for the deletions and one for the copies.
Specifying --delete-during will delete files while checking and uploading files.  This is the fastest option and uses the least memory.
Specifying --delete-after (the default value) will delay deletion of files until all new/updated files have been successfully transferred. The files to be deleted are collected in the copy pass then deleted after the copy pass has completed successfully.  The files to be deleted are held in memory so this mode may use more memory.  This is the safest mode as it will only delete files if there have been no errors subsequent to that.  If there have been errors before the deletions start then you will get the message not deleting files as there were IO errors.
--fast-list When doing anything which involves a directory listing (e.g.  sync, copy, ls - in fact nearly every command), rclone normally lists a directory and processes it before using more directory lists to process any subdirectories.  This can be parallelised and works very quickly using the least amount of memory.
However, some remotes have a way of listing all files beneath a directory in one (or a small number) of transactions.  These tend to be the bucket based remotes (e.g.  S3, B2, GCS, Swift, Hubic).
If you use the --fast-list flag then rclone will use this method for listing directories.  This will have the following consequences for the listing:
• It will use fewer transactions (important if you pay for them)
• It will use more memory.  Rclone has to load the whole listing into memory.
• It may be faster because it uses fewer transactions
• It may be slower because it can't be parallelized
rclone should always give identical results with and without --fast-list.
If you pay for transactions and can fit your entire sync listing into memory then --fast-list is recommended.  If you have a very big sync to do then don't use --fast-list otherwise you will run out of memory.
If you use --fast-list on a remote which doesn't support it, then rclone will just ignore it.
--timeout=TIME This sets the IO idle timeout.  If a transfer has started but then becomes idle for this long it is considered broken and disconnected.
The default is 5m.  Set to 0 to disable.
--transfers=N The number of file transfers to run in parallel.  It can sometimes be useful to set this to a smaller number if the remote is giving a lot of timeouts or bigger if you have lots of bandwidth and a fast remote.
The default is to run 4 file transfers in parallel.
-u, --update This forces rclone to skip any files which exist on the destination and have a modified time that is newer than the source file.
This can be useful when transferring to a remote which doesn't support mod times directly (or when using --use-server-modtime to avoid extra API calls) as it is more accurate than a --size-only check and faster than using --checksum.
If an existing destination file has a modification time equal (within the computed modify window precision) to the source file's, it will be updated if the sizes are different.  If --checksum is set then rclone will update the destination if the checksums differ too.
If an existing destination file is older than the source file then it will be updated if the size or checksum differs from the source file.
On remotes which don't support mod time directly (or when using --use-server-modtime) the time checked will be the uploaded time.  This means that if uploading to one of these remotes, rclone will skip any files which exist on the destination and have an uploaded time that is newer than the modification time of the source file.
--use-mmap If this flag is set then rclone will use anonymous memory allocated by mmap on Unix based platforms and VirtualAlloc on Windows for its transfer buffers (size controlled by --buffer-size).  Memory allocated like this does not go on the Go heap and can be returned to the OS immediately when it is finished with.
If this flag is not set then rclone will allocate and free the buffers using the Go memory allocator which may use more memory as memory pages are returned less aggressively to the OS.
It is possible this does not work well on all platforms so it is disabled by default; in the future it may be enabled by default.
--use-server-modtime Some object-store backends (e.g, Swift, S3) do not preserve file modification times (modtime).  On these backends, rclone stores the original modtime as additional metadata on the object.  By default it will make an API call to retrieve the metadata when the modtime is needed by an operation.
Use this flag to disable the extra API call and rely instead on the server's modified time.  In cases such as a local to remote sync using --update, knowing the local file is newer than the time it was last uploaded to the remote is sufficient.  In those cases, this flag can speed up the process and reduce the number of API calls necessary.
Using this flag on a sync operation without also using --update would cause all files modified at any time other than the last upload time to be uploaded again, which is probably not what you want.
-v, -vv, --verbose With -v rclone will tell you about each file that is transferred and a small number of significant events.
With -vv rclone will become very verbose telling you about every file it considers and transfers.  Please send bug reports with a log with this setting.
-V, --version Prints the version number
SSL/TLS options The outgoing SSL/TLS connections rclone makes can be controlled with these options.  For example this can be very useful with the HTTP or WebDAV backends.  Rclone HTTP servers have their own set of configuration for SSL/TLS which you can find in their documentation.
--ca-cert string This loads the PEM encoded certificate authority certificate and uses it to verify the certificates of the servers rclone connects to.
If you have generated certificates signed with a local CA then you will need this flag to connect to servers using those certificates.
--client-cert string This loads the PEM encoded client side certificate.
This is used for mutual TLS authentication (https://en.wikipedia.org/wiki/Mutual_authentication).
The --client-key flag is required too when using this.
--client-key string This loads the PEM encoded client side private key used for mutual TLS authentication.  Used in conjunction with --client-cert.
--no-check-certificate=true/false --no-check-certificate controls whether a client verifies the server's certificate chain and host name.  If --no-check-certificate is true, TLS accepts any certificate presented by the server and any host name in that certificate.  In this mode, TLS is susceptible to man-in-the-middle attacks.
This option defaults to false.
This should be used only for testing.
Configuration Encryption Your configuration file contains information for logging in to your cloud services.  This means that you should keep your rclone.conf file in a secure location.
If you are in an environment where that isn't possible, you can add a password to your configuration.  This means that you will have to supply the password every time you start rclone.
To add a password to your rclone configuration, execute rclone config.
>rclone config Current remotes:
e) Edit existing remote n) New remote d) Delete remote s) Set configuration password q) Quit config e/n/d/s/q>
Go into s, Set configuration password:
e/n/d/s/q> s Your configuration is not encrypted. If you add a password, you will protect your login information to cloud services. a) Add Password q) Quit to main menu a/q> a Enter NEW configuration password: password: Confirm NEW password: password: Password set Your configuration is encrypted. c) Change Password u) Unencrypt configuration q) Quit to main menu c/u/q>
Your configuration is now encrypted, and every time you start rclone you will have to supply the password.  See below for details.  In the same menu, you can change the password or completely remove encryption from your configuration.
There is no way to recover the configuration if you lose your password.
rclone uses nacl secretbox (https://godoc.org/golang.org/x/crypto/nacl/secretbox) which in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your configuration with secret-key cryptography.  The password is SHA-256 hashed, which produces the key for secretbox.  The hashed password is not stored.
While this provides very good security, we do not recommend storing your encrypted rclone configuration in public if it contains sensitive information, maybe except if you use a very strong password.
If it is safe in your environment, you can set the RCLONE_CONFIG_PASS environment variable to contain your password, in which case it will be used for decrypting the configuration.
You can set this for a session from a script.  For unix like systems save this to a file called set-rclone-password:
#!/bin/echo Source this file don't run it
read -s RCLONE_CONFIG_PASS export RCLONE_CONFIG_PASS
Then source the file when you want to use it.  From the shell you would do source set-rclone-password.  It will then ask you for the password and set it in the environment variable.
An alternate means of supplying the password is to provide a script which will retrieve the password and print on standard output.  This script should have a fully specified path name and not rely on any environment variables.  The script is supplied either via --password-command="..." command line argument or via the RCLONE_PASSWORD_COMMAND environment variable.
One useful example of this is using the passwordstore application to retrieve the password:
export RCLONE_PASSWORD_COMMAND="pass rclone/config"
If the passwordstore password manager holds the password for the rclone configuration, using the script method means the password is primarily protected by the passwordstore system, and is never embedded in the clear in scripts, nor available for examination using the standard commands available.  It is quite possible with long running rclone sessions for copies of passwords to be innocently captured in log files or terminal scroll buffers, etc.  Using the script method of supplying the password enhances the security of the config password considerably.
If you are running rclone inside a script, unless you are using the --password-command method, you might want to disable password prompts. To do that, pass the parameter --ask-password=false to rclone.  This will make rclone fail instead of asking for a password if RCLONE_CONFIG_PASS doesn't contain a valid password, and --password-command has not been supplied.
Whenever running commands that may be affected by options in a configuration file, rclone will look for an existing file according to the rules described above, and load any it finds.  If an encrypted file is found, this includes decrypting it, with the possible consequence of a password prompt.  When executing a command line that you know are not actually using anything from such a configuration file, you can avoid it being loaded by overriding the location, e.g.  with one of the documented special values for memory-only configuration.  Since only backend options can be stored in configuration files, this is normally unnecessary for commands that do not operate on backends, e.g.  genautocomplete. However, it will be relevant for commands that do operate on backends in general, but are used without referencing a stored remote, e.g.  listing local filesystem paths, or connection strings: rclone --config="" ls .
Developer options These options are useful when developing or debugging rclone.  There are also some more remote specific options which aren't documented here which are used for testing.  These start with remote name e.g. --drive-test-option - see the docs for the remote in question.
--cpuprofile=FILE Write CPU profile to file.  This can be analysed with go tool pprof.
--dump flag,flag,flag The --dump flag takes a comma separated list of flags to dump info about.
Note that some headers including Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect.  In this case the body of the request will be gunzipped before showing it.
The available flags are:
--dump headers Dump HTTP headers with Authorization: lines removed.  May still contain sensitive info.  Can be very verbose.  Useful for debugging only.
Use --dump auth if you do want the Authorization: headers.
--dump bodies Dump HTTP headers and bodies - may contain sensitive info.  Can be very verbose.  Useful for debugging only.
Note that the bodies are buffered in memory so don't use this for enormous files.
--dump requests Like --dump bodies but dumps the request bodies and the response headers. Useful for debugging download problems.
--dump responses Like --dump bodies but dumps the response bodies and the request headers. Useful for debugging upload problems.
--dump auth Dump HTTP headers - will contain sensitive info such as Authorization: headers - use --dump headers to dump without Authorization: headers.  Can be very verbose.  Useful for debugging only.
--dump filters Dump the filters to the output.  Useful to see exactly what include and exclude options are filtering on.
--dump goroutines This dumps a list of the running go-routines at the end of the command to standard output.
--dump openfiles This dumps a list of the open files at the end of the command.  It uses the lsof command to do that so you'll need that installed to use it.
--memprofile=FILE Write memory profile to file.  This can be analysed with go tool pprof.
Filtering For the filtering options
• --delete-excluded
• --filter
• --filter-from
• --exclude
• --exclude-from
• --include
• --include-from
• --files-from
• --files-from-raw
• --min-size
• --max-size
• --min-age
• --max-age
• --dump filters
See the filtering section (https://rclone.org/filtering/).
Remote control For the remote control options and for instructions on how to remote control rclone
• --rc
• and anything starting with --rc-
See the remote control section (https://rclone.org/rc/).
Logging rclone has 4 levels of logging, ERROR, NOTICE, INFO and DEBUG.
By default, rclone logs to standard error.  This means you can redirect standard error and still see the normal output of rclone commands (e.g. rclone ls).
By default, rclone will produce Error and Notice level messages.
If you use the -q flag, rclone will only produce Error messages.
If you use the -v flag, rclone will produce Error, Notice and Info messages.
If you use the -vv flag, rclone will produce Error, Notice, Info and Debug messages.
You can also control the log levels with the --log-level flag.
If you use the --log-file=FILE option, rclone will redirect Error, Info and Debug messages along with standard error to FILE.
If you use the --syslog flag then rclone will log to syslog and the --syslog-facility control which facility it uses.
Rclone prefixes all log messages with their level in capitals, e.g.  INFO which makes it easy to grep the log file for different kinds of information.
Exit Code If any errors occur during the command execution, rclone will exit with a non-zero exit code.  This allows scripts to detect when rclone operations have failed.
During the startup phase, rclone will exit immediately if an error is detected in the configuration.  There will always be a log message immediately before exiting.
When rclone is running it will accumulate errors as it goes along, and only exit with a non-zero exit code if (after retries) there were still failed transfers.  For every error counted there will be a high priority log message (visible with -q) showing the message and which file caused the problem.  A high priority message is also shown when starting a retry so the user can see that any previous error messages may not be valid after the retry.  If rclone has done a retry it will log a high priority message if the retry was successful.
List of exit codes • 0 - success
• 1 - Syntax or usage error
• 2 - Error not otherwise categorised
• 3 - Directory not found
• 4 - File not found
• 5 - Temporary error (one that more retries might fix) (Retry errors)
• 6 - Less serious errors (like 461 errors from dropbox) (NoRetry errors)
• 7 - Fatal error (one that more retries won't fix, like account suspended) (Fatal errors)
• 8 - Transfer exceeded - limit set by --max-transfer reached
• 9 - Operation successful, but no files transferred
Environment Variables Rclone can be configured entirely using environment variables.  These can be used to set defaults for options or config file entries.
Options Every option in rclone can have its default set by environment variable.
To find the name of the environment variable, first, take the long option name, strip the leading --, change - to _, make upper case and prepend RCLONE_.
For example, to always set --stats 5s, set the environment variable RCLONE_STATS=5s.  If you set stats on the command line this will override the environment variable setting.
Or to always use the trash in drive --drive-use-trash, set RCLONE_DRIVE_USE_TRASH=true.
The same parser is used for the options and the environment variables so they take exactly the same form.
The options set by environment variables can be seen with the -vv flag, e.g.  rclone version -vv.
Config file You can set defaults for values in the config file on an individual remote basis.  The names of the config items are documented in the page for each backend.
To find the name of the environment variable, you need to set, take RCLONE_CONFIG_ + name of remote + _ + name of config file option and make it all uppercase.
For example, to configure an S3 remote named mys3: without a config file (using unix ways of setting environment variables):
$ export RCLONE_CONFIG_MYS3_TYPE=s3 $ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX $ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX $ rclone lsd mys3: -1 2016-09-21 12:54:21        -1 my-bucket $ rclone listremotes | grep mys3 mys3:
Note that if you want to create a remote using environment variables you must create the ..._TYPE variable as above.
Note that the name of a remote created using environment variable is case insensitive, in contrast to regular remotes stored in config file as documented above.  You must write the name in uppercase in the environment variable, but as seen from example above it will be listed and can be accessed in lowercase, while you can also refer to the same remote in uppercase:
$ rclone lsd mys3: -1 2016-09-21 12:54:21        -1 my-bucket $ rclone lsd MYS3: -1 2016-09-21 12:54:21        -1 my-bucket
Note that you can only set the options of the immediate backend, so RCLONE_CONFIG_MYS3CRYPT_ACCESS_KEY_ID has no effect, if myS3Crypt is a crypt remote based on an S3 remote.  However RCLONE_S3_ACCESS_KEY_ID will set the access key of all remotes using S3, including myS3Crypt.
Note also that now rclone has connection strings, it is probably easier to use those instead which makes the above example
rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
Precedence The various different methods of backend configuration are read in this order and the first one with a value is used.
• Parameters in connection strings, e.g.  myRemote,skip_links:
• Flag values as supplied on the command line, e.g.  --skip-links
• Remote specific environment vars, e.g. RCLONE_CONFIG_MYREMOTE_SKIP_LINKS (see above).
• Backend specific environment vars, e.g.  RCLONE_LOCAL_SKIP_LINKS.
• Backend generic environment vars, e.g.  RCLONE_SKIP_LINKS.
• Config file, e.g.  skip_links = true.
• Default values, e.g.  false - these can't be changed.
So if both --skip-links is supplied on the command line and an environment variable RCLONE_LOCAL_SKIP_LINKS is set, the command line flag will take preference.
The backend configurations set by environment variables can be seen with the -vv flag, e.g.  rclone about myRemote: -vv.
For non backend configuration the order is as follows:
• Flag values as supplied on the command line, e.g.  --stats 5s.
• Environment vars, e.g.  RCLONE_STATS=5s.
• Default values, e.g.  1m - these can't be changed.
Other environment variables • RCLONE_CONFIG_PASS set to contain your config file password (see Configuration Encryption section)
• HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions thereof).
• HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.
• The environment values may be either a complete URL or a "host[:port]" for, in which case the "http" scheme is assumed.
• USER and LOGNAME values are used as fallbacks for current username. The primary method for looking up username is OS-specific: Windows API on Windows, real user ID in /etc/passwd on Unix systems.  In the documentation the current username is simply referred to as $USER.
• RCLONE_CONFIG_DIR - rclone sets this variable for use in config files and sub processes to point to the directory holding the config file.
The options set by environment variables can be seen with the -vv and --log-level=DEBUG flags, e.g.  rclone version -vv.
Configuring rclone on a remote / headless machine Some of the configurations (those involving oauth2) require an Internet connected web browser.
If you are trying to set rclone up on a remote or headless box with no browser available on it (e.g.  a NAS or a server in a datacenter) then you will need to use an alternative means of configuration.  There are two ways of doing it, described below.
Configuring using rclone authorize On the headless box run rclone config but answer N to the Use auto config? question.
... Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes (default) n) No y/n> n For this to work, you will need rclone available on a machine that has a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone version recommended):
rclone authorize "amazon cloud drive"
Then paste the result below: result>
Then on your main desktop machine
rclone authorize "amazon cloud drive" If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth Log in and authorize rclone for access Waiting for code... Got code Paste the following into your remote machine ---> SECRET_TOKEN <---End paste
Then back to the headless box, paste in the code
result> SECRET_TOKEN -------------------- [acd12] client_id = client_secret = token = SECRET_TOKEN -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d>
Configuring by copying the config file Rclone stores all of its config in a single configuration file.  This can easily be copied to configure a remote rclone.
So first configure rclone on your desktop machine with
to set up the config file.
Find the config file by running rclone config file, for example
$ rclone config file Configuration file is stored at: /home/user/.rclone.conf
Now transfer it to the remote box (scp, cut paste, ftp, sftp, etc.) and place it in the correct place (use rclone config file on the remote box to find out where).
Filtering, includes and excludes Filter flags determine which files rclone sync, move, ls, lsl, md5sum, sha1sum, size, delete, check and similar commands apply to.
They are specified in terms of path/file name patterns; path/file lists; file age and size, or presence of a file in a directory.  Bucket based remotes without the concept of directory apply filters to object key, age and size in an analogous way.
Rclone purge does not obey filters.
To test filters without risk of damage to data, apply them to rclone ls, or with the --dry-run and -vv flags.
Rclone filter patterns can only be used in filter command line options, not in the specification of a remote.
E.g.  rclone copy "remote:dir*.jpg" /path/to/dir does not have a filter effect.  rclone copy remote:dir /path/to/dir --include "*.jpg" does.
Important Avoid mixing any two of --include..., --exclude... or --filter... flags in an rclone command.  The results may not be what you expect.  Instead use a --filter... flag.
Patterns for matching path/file names Pattern syntax Rclone matching rules follow a glob style:
*         matches any sequence of non-separator (/) characters **        matches any sequence of characters including / separators ?         matches any single non-separator (/) character [ [ ! ] { character-range } ] character class (must be non-empty) { pattern-list } pattern alternatives c         matches character c (c != *, **, ?, \, [, {, }) \c        matches reserved character c (c = *, **, ?, \, [, {, })
character-range:
c         matches character c (c != \, -, ]) \c        matches reserved character c (c = \, -, ]) lo - hi   matches character c for lo <= c <= hi
pattern-list:
pattern { , pattern } comma-separated (without spaces) patterns
character classes (see Go regular expression reference (https://golang.org/pkg/regexp/syntax/)) include:
Named character classes (e.g. [\d], [^\d], [\D], [^\D]) Perl character classes (e.g. \s, \S, \w, \W) ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
If the filter pattern starts with a / then it only matches at the top level of the directory tree, relative to the root of the remote (not necessarily the root of the drive).  If it does not start with / then it is matched starting at the end of the path/file name but it only matches a complete path element - it must match from a / separator or the beginning of the path/file.
file.jpg   - matches "file.jpg" - matches "directory/file.jpg" - doesn't match "afile.jpg" - doesn't match "directory/afile.jpg" /file.jpg  - matches "file.jpg" in the root directory of the remote - doesn't match "afile.jpg" - doesn't match "directory/file.jpg"
The top level of the remote may not be the top level of the drive.
E.g.  for a Microsoft Windows local directory structure
F: ├── bkp ├── data │   ├── excl │   │   ├── 123.jpg │   │   └── 456.jpg │   ├── incl │   │   └── document.pdf
To copy the contents of folder data into folder bkp excluding the contents of subfolder exclthe following command treats F:\data and F:\bkp as top level for filtering.
rclone copy F:\data\ F:\bkp\ --exclude=/excl/**
Important Use / in path/file name patterns and not \ even if running on Microsoft Windows.
Simple patterns are case sensitive unless the --ignore-case flag is used.
Without --ignore-case (default)
potato - matches "potato" - doesn't match "POTATO"
With --ignore-case
potato - matches "potato" - matches "POTATO"
How filter rules are applied to files Rclone path/file name filters are made up of one or more of the following flags:
There can be more than one instance of individual flags.
Rclone internally uses a combined list of all the include and exclude rules.  The order in which rules are processed can influence the result of the filter.
All flags of the same type are processed together in the order above, regardless of what order the different types of flags are included on the command line.
Multiple instances of the same flag are processed from left to right according to their position in the command line.
To mix up the order of processing includes and excludes use --filter... flags.
Within --include-from, --exclude-from and --filter-from flags rules are processed from top to bottom of the referenced file.
If there is an --include or --include-from flag specified, rclone implies a - ** rule which it adds to the bottom of the internal rule list. Specifying a + rule with a --filter... flag does not imply that rule.
Each path/file name passed through rclone is matched against the combined filter list.  At first match to a rule the path/file name is included or excluded and no further filter rules are processed for that path/file.
If rclone does not find a match, after testing against all rules (including the implied rule if appropriate), the path/file name is included.
Any path/file included at that stage is processed by the rclone command.
--files-from and --files-from-raw flags over-ride and cannot be combined with other filter options.
To see the internal combined rule list, in regular expression form, for a command add the --dump filters flag.  Running an rclone command with --dump filters and -vv flags lists the internal filter elements and shows how they are applied to each source path/file.  There is not currently a means provided to pass regular expression filter options into rclone directly though character class filter rules contain character classes. Go regular expression reference (https://golang.org/pkg/regexp/syntax/)
How filter rules are applied to directories Rclone commands are applied to path/file names not directories.  The entire contents of a directory can be matched to a filter by the pattern directory/* or recursively by directory/**.
Directory filter rules are defined with a closing / separator.
E.g.  /directory/subdirectory/ is an rclone directory filter rule.
Rclone commands can use directory filter rules to determine whether they recurse into subdirectories.  This potentially optimises access to a remote by avoiding listing unnecessary directories.  Whether optimisation is desirable depends on the specific filter rules and source remote content.
Directory recursion optimisation occurs if either:
• A source remote does not support the rclone ListR primitive.  local, sftp, Microsoft OneDrive and WebDav do not support ListR.  Google Drive and most bucket type storage do.  Full list (https://rclone.org/overview/#optional-features)
• On other remotes (those that support ListR), if the rclone command is not naturally recursive, and provided it is not run with the --fast-list flag.  ls, lsf -R and size are naturally recursive but sync, copy and move are not.
• Whenever the --disable ListR flag is applied to an rclone command.
Rclone commands imply directory filter rules from path/file filter rules. To view the directory filter rules rclone has implied for a command specify the --dump filters flag.
E.g.  for an include rule
/a/*.jpg
Rclone implies the directory include rule
Directory filter rules specified in an rclone command can limit the scope of an rclone command but path/file filters still have to be specified.
E.g.  rclone ls remote: --include /directory/ will not match any files. Because it is an --include option the --exclude ** rule is implied, and the /directory/ pattern serves only to optimise access to the remote by ignoring everything outside of that directory.
E.g.  rclone ls remote: --filter-from filter-list.txt with a file filter-list.txt:
- /dir1/ - /dir2/ + *.pdf - **
All files in directories dir1 or dir2 or their subdirectories are completely excluded from the listing.  Only files of suffix pdf in the root of remote: or its subdirectories are listed.  The - ** rule prevents listing of any path/files not previously matched by the rules above.
Option exclude-if-present creates a directory exclude rule based on the presence of a file in a directory and takes precedence over other rclone directory filter rules.
When using pattern list syntax, if a pattern item contains either / or **, then rclone will not able to imply a directory filter rule from this pattern list.
{dir1/**,dir2/**}
Rclone will match files below directories dir1 or dir2 only, but will not be able to use this filter to exclude a directory dir3 from being traversed.
Directory recursion optimisation may affect performance, but normally not the result.  One exception to this is sync operations with option --create-empty-src-dirs, where any traversed empty directories will be created.  With the pattern list example {dir1/**,dir2/**} above, this would create an empty directory dir3 on destination (when it exists on source).  Changing the filter to {dir1,dir2}/**, or splitting it into two include rules --include dir1/** --include dir2/**, will match the same files while also filtering directories, with the result that an empty directory dir3 will no longer be created.
--exclude - Exclude files matching pattern Excludes path/file names from an rclone command based on a single exclude rule.
This flag can be repeated.  See above for the order filter flags are processed in.
--exclude should not be used with --include, --include-from, --filter or --filter-from flags.
--exclude has no effect when combined with --files-from or --files-from-raw flags.
E.g.  rclone ls remote: --exclude *.bak excludes all .bak files from listing.
E.g.  rclone size remote: "--exclude /dir/**" returns the total size of all files on remote: excluding those in root directory dir and sub directories.
E.g.  on Microsoft Windows rclone ls remote: --exclude "*\[{JP,KR,HK}\]*" lists the files in remote: with [JP] or [KR] or [HK] in their name. Quotes prevent the shell from interpreting the \ characters.\ characters escape the [ and ] so an rclone filter treats them literally rather than as a character-range.  The { and } define an rclone pattern list.  For other operating systems single quotes are required ie rclone ls remote: --exclude '*\[{JP,KR,HK}\]*'
--exclude-from - Read exclude patterns from file Excludes path/file names from an rclone command based on rules in a named file.  The file contains a list of remarks and pattern rules.
For an example exclude-file.txt:
# a sample exclude rule file *.bak file2.jpg
rclone ls remote: --exclude-from exclude-file.txt lists the files on remote: except those named file2.jpg or with a suffix .bak.  That is equivalent to rclone ls remote: --exclude file2.jpg --exclude "*.bak".
The --exclude-from flag is useful where multiple exclude filter rules are applied to an rclone command.
--exclude-from should not be used with --include, --include-from, --filter or --filter-from flags.
--exclude-from has no effect when combined with --files-from or --files-from-raw flags.
--exclude-from followed by - reads filter rules from standard input.
--include - Include files matching pattern Adds a single include rule based on path/file names to an rclone command.
--include has no effect when combined with --files-from or --files-from-raw flags.
--include implies --exclude ** at the end of an rclone internal filter list.  Therefore if you mix --include and --include-from flags with --exclude, --exclude-from, --filter or --filter-from, you must use include rules for all the files you want in the include statement.  For more flexibility use the --filter-from flag.
E.g.  rclone ls remote: --include "*.{png,jpg}" lists the files on remote: with suffix .png and .jpg.  All other files are excluded.
E.g.  multiple rclone copy commands can be combined with --include and a pattern-list.
rclone copy /vol1/A remote:A rclone copy /vol1/B remote:B
is equivalent to:
rclone copy /vol1 remote: --include "{A,B}/**"
E.g.  rclone ls remote:/wheat --include "??[^[:punct:]]*" lists the files remote: directory wheat (and subdirectories) whose third character is not punctuation.  This example uses an ASCII character class (https://golang.org/pkg/regexp/syntax/).
--include-from - Read include patterns from file Adds path/file names to an rclone command based on rules in a named file. The file contains a list of remarks and pattern rules.
For an example include-file.txt:
# a sample include rule file *.jpg file2.avi
rclone ls remote: --include-from include-file.txt lists the files on remote: with name file2.avi or suffix .jpg.  That is equivalent to rclone ls remote: --include file2.avi --include "*.jpg".
The --include-from flag is useful where multiple include filter rules are applied to an rclone command.
--include-from implies --exclude ** at the end of an rclone internal filter list.  Therefore if you mix --include and --include-from flags with --exclude, --exclude-from, --filter or --filter-from, you must use include rules for all the files you want in the include statement.  For more flexibility use the --filter-from flag.
--filter - Add a file-filtering rule Specifies path/file names to an rclone command, based on a single include or exclude rule, in + or - format.
--filter + differs from --include.  In the case of --include rclone implies an --exclude * rule which it adds to the bottom of the internal rule list.  --filter...+ does not imply that rule.
--filter has no effect when combined with --files-from or --files-from-raw flags.
--filter should not be used with --include, --include-from, --exclude or --exclude-from flags.
E.g.  rclone ls remote: --filter "- *.bak" excludes all .bak files from a list of remote:.
--filter-from - Read filtering patterns from a file Adds path/file names to an rclone command based on rules in a named file. The file contains a list of remarks and pattern rules.  Include rules start with + and exclude rules with -.  ! clears existing rules.  Rules are processed in the order they are defined.
Arrange the order of filter rules with the most restrictive first and work down.
E.g.  for filter-file.txt:
# a sample filter rule file - secret*.jpg + *.jpg + *.png + file2.avi - /dir/Trash/** + /dir/** # exclude everything else - *
rclone ls remote: --filter-from filter-file.txt lists the path/files on remote: including all jpg and png files, excluding any matching secret*.jpg and including file2.avi.  It also includes everything in the directory dir at the root of remote, except remote:dir/Trash which it excludes.  Everything else is excluded.
E.g.  for an alternative filter-file.txt:
- secret*.jpg + *.jpg + *.png + file2.avi - *
Files file1.jpg, file3.png and file2.avi are listed whilst secret17.jpg and files without the suffix .jpgor.png` are excluded.
+ *.jpg + *.gif ! + 42.doc - *
Only file 42.doc is listed.  Prior rules are cleared by the !.
--files-from - Read list of source-file names Adds path/files to an rclone command from a list in a named file.  Rclone processes the path/file names in the order of the list, and no others.
Other filter flags (--include, --include-from, --exclude, --exclude-from, --filter and --filter-from) are ignored when --files-from is used.
--files-from expects a list of files as its input.  Leading or trailing whitespace is stripped from the input lines.  Lines starting with # or ; are ignored.
Rclone commands with a --files-from flag traverse the remote, treating the names in --files-from as a set of filters.
If the --no-traverse and --files-from flags are used together an rclone command does not traverse the remote.  Instead it addresses each path/file named in the file individually.  For each path/file name, that requires typically 1 API call.  This can be efficient for a short --files-from list and a remote containing many files.
Rclone commands do not error if any names in the --files-from file are missing from the source remote.
The --files-from flag can be repeated in a single rclone command to read path/file names from more than one file.  The files are read from left to right along the command line.
Paths within the --files-from file are interpreted as starting with the root specified in the rclone command.  Leading / separators are ignored. See --files-from-raw if you need the input to be processed in a raw manner.
E.g.  for a file files-from.txt:
# comment file1.jpg subdir/file2.jpg
rclone copy --files-from files-from.txt /home/me/pics remote:pics copies the following, if they exist, and only those files.
/home/me/pics/file1.jpg        → remote:pics/file1.jpg /home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
E.g.  to copy the following files referenced by their absolute paths:
/home/user1/42 /home/user1/dir/ford /home/user2/prefect
First find a common subdirectory - in this case /home and put the remaining files in files-from.txt with or without leading /, e.g.
user1/42 user1/dir/ford user2/prefect
Then copy these to a remote:
rclone copy --files-from files-from.txt /home remote:backup
The three files are transferred as follows:
/home/user1/42       → remote:backup/user1/important /home/user1/dir/ford → remote:backup/user1/dir/file /home/user2/prefect  → remote:backup/user2/stuff
Alternatively if / is chosen as root files-from.txt will be:
The copy command will be:
rclone copy --files-from files-from.txt / remote:backup
Then there will be an extra home directory on the remote:
/home/user1/42       → remote:backup/home/user1/42 /home/user1/dir/ford → remote:backup/home/user1/dir/ford /home/user2/prefect  → remote:backup/home/user2/prefect
--files-from-raw - Read list of source-file names without any processing This flag is the same as --files-from except that input is read in a raw manner.  Lines with leading / trailing whitespace, and lines starting with ; or # are read without any processing.  rclone lsf (https://rclone.org/commands/rclone_lsf/) has a compatible format that can be used to export file lists from remotes for input to --files-from-raw.
--ignore-case - make searches case insensitive By default rclone filter patterns are case sensitive.  The --ignore-case flag makes all of the filters patterns on the command line case insensitive.
E.g.  --include "zaphod.txt" does not match a file Zaphod.txt.  With --ignore-case a match is made.
Quoting shell metacharacters Rclone commands with filter patterns containing shell metacharacters may not as work as expected in your shell and may require quoting.
E.g.  linux, OSX (* metacharacter)
• --include \*.jpg
• --include '*.jpg'
• --include='*.jpg'
Microsoft Windows expansion is done by the command, not shell, so --include *.jpg does not require quoting.
If the rclone error Command .... needs .... arguments maximum: you provided .... non flag arguments: is encountered, the cause is commonly spaces within the name of a remote or flag value.  The fix then is to quote values containing spaces.
Other filters --min-size - Don't transfer any file smaller than this Controls the minimum size file within the scope of an rclone command. Default units are KiB but abbreviations K, M, G, T or P are valid.
E.g.  rclone ls remote: --min-size 50k lists files on remote: of 50 KiB size or larger.
--max-size - Don't transfer any file larger than this Controls the maximum size file within the scope of an rclone command. Default units are KiB but abbreviations K, M, G, T or P are valid.
E.g.  rclone ls remote: --max-size 1G lists files on remote: of 1 GiB size or smaller.
--max-age - Don't transfer any file older than this Controls the maximum age of files within the scope of an rclone command. Default units are seconds or the following abbreviations are valid:
• ms - Milliseconds
• s - Seconds
• m - Minutes
• h - Hours
• d - Days
• w - Weeks
• M - Months
• y - Years
--max-age can also be specified as an absolute time in the following formats:
• RFC3339 - e.g.  2006-01-02T15:04:05Z or 2006-01-02T15:04:05+07:00
• ISO8601 Date and time, local timezone - 2006-01-02T15:04:05
• ISO8601 Date and time, local timezone - 2006-01-02 15:04:05
• ISO8601 Date - 2006-01-02 (YYYY-MM-DD)
--max-age applies only to files and not to directories.
E.g.  rclone ls remote: --max-age 2d lists files on remote: of 2 days old or less.
--min-age - Don't transfer any file younger than this Controls the minimum age of files within the scope of an rclone command. (see --max-age for valid formats)
--min-age applies only to files and not to directories.
E.g.  rclone ls remote: --min-age 2d lists files on remote: of 2 days old or more.
Other flags --delete-excluded - Delete files on dest excluded from sync Important this flag is dangerous to your data - use with --dry-run and -v first.
In conjunction with rclone sync, --delete-excluded deletes any files on the destination which are excluded from the command.
E.g.  the scope of rclone sync -i A: B: can be restricted:
rclone --min-size 50k --delete-excluded sync A: B:
All files on B: which are less than 50 KiB are deleted because they are excluded from the rclone sync command.
--dump filters - dump the filters to the output Dumps the defined filters to standard output in regular expression format.
Useful for debugging.
Exclude directory based on a file The --exclude-if-present flag controls whether a directory is within the scope of an rclone command based on the presence of a named file within it.
This flag has a priority over other filter flags.
E.g.  for the following directory structure:
dir1/file1 dir1/dir2/file2 dir1/dir2/dir3/file3 dir1/dir2/dir3/.ignore
The command rclone ls --exclude-if-present .ignore dir1 does not list dir3, file3 or .ignore.
--exclude-if-present can only be used once in an rclone command.
Common pitfalls The most frequent filter support issues on the rclone forum (https://forum.rclone.org/) are:
• Not using paths relative to the root of the remote
• Not using / to match from the root of a remote
• Not using ** to match the contents of a directory
GUI (Experimental) Rclone can serve a web based GUI (graphical user interface).  This is somewhat experimental at the moment so things may be subject to change.
Run this command in a terminal and rclone will download and then display the GUI in a web browser.
rclone rcd --rc-web-gui
This will produce logs like this and rclone needs to continue to run to serve the GUI: