Repository: pkolaczk/fclones (language: rust)

Issue:
Problem with finding files on synology NAS share mounted as CIFS volume under Linux
Hello Piotr,

Thanks for your work on fclones,

I had a plan to use for deduplication for files on my NAS, however I encountered a strange problem.

Please find directory contents on my nas:
```bash
'IMG_20210416_204824 (1).jpg'*  'IMG_20210416_204830 (2).jpg'*   IMG_20210416_204845.jpg*       'IMG_20210416_223757 (1).jpg'*  'IMG_20210416_224002 (2).jpg'*   IMG_20210416_224021.jpg*        IMG_20210416_232550.jpg*
'IMG_20210416_204824 (2).jpg'*   IMG_20210416_204830.jpg*       'IMG_20210416_204847 (1).jpg'*  'IMG_20210416_223757 (2).jpg'*   IMG_20210416_224002.jpg*       'IMG_20210416_224022 (1).jpg'*   Thumbs.db*
 IMG_20210416_204824.jpg*       'IMG_20210416_204832 (1).jpg'*  'IMG_20210416_204847 (2).jpg'*   IMG_20210416_223757.jpg*       'IMG_20210416_224015 (1).jpg'*  'IMG_20210416_224022 (2).jpg'*   VID_20210416_224914.mp4*
'IMG_20210416_204827 (1).jpg'*  'IMG_20210416_204832 (2).jpg'*   IMG_20210416_204847.jpg*       'IMG_20210416_223758 (1).jpg'*  'IMG_20210416_224015 (2).jpg'*   IMG_20210416_224022.jpg*        VID_20210416_232553.mp4*
'IMG_20210416_204827 (2).jpg'*   IMG_20210416_204832.jpg*       'IMG_20210416_223755 (1).jpg'*  'IMG_20210416_223758 (2).jpg'*   IMG_20210416_224015.jpg*       'IMG_20210416_224107 (1).jpg'*
 IMG_20210416_204827.jpg*       'IMG_20210416_204845 (1).jpg'*  'IMG_20210416_223755 (2).jpg'*   IMG_20210416_223758.jpg*       'IMG_20210416_224021 (1).jpg'*  'IMG_20210416_224107 (2).jpg'*
'IMG_20210416_204830 (1).jpg'*  'IMG_20210416_204845 (2).jpg'*   IMG_20210416_223755.jpg*       'IMG_20210416_224002 (1).jpg'*  'IMG_20210416_224021 (2).jpg'*   IMG_20210416_224107.jpg*
```
Please find that files with `(1)` or `(2)` in their names are duplicates for sure, I confirmed this by `md5sum` cmd - they simply have the same size.

Directory is mounted as `type cifs (rw,relatime,vers=3.1.1,cache=strict,username=agnieszka,uid=1000,forceuid,gid=1000,forcegid,addr=10.0.0.10,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1)`

fclones --version `0.10.2`
linux version `5.4.102-rt53-MANJARO`

While I'm in this directory with duplicates I use the following fclones cmd: `fclones .`

And I got report:

```bash
[2021-04-17 20:51:17.333] fclones:  info: Started
[2021-04-17 20:51:18.125] fclones:  info: Scanned 1 file entries
[2021-04-17 20:51:18.125] fclones:  info: Found 0 (0 B) files matching selection criteria
[2021-04-17 20:51:18.125] fclones:  info: Found 0 (0 B) candidates after grouping by size
[2021-04-17 20:51:18.126] fclones:  info: Found 0 (0 B) candidates after grouping by paths and file identifiers
[2021-04-17 20:51:18.126] fclones:  info: Found 0 (0 B) candidates after grouping by prefix
[2021-04-17 20:51:18.126] fclones:  info: Found 0 (0 B) candidates after grouping by suffix
[2021-04-17 20:51:18.126] fclones:  info: Found 0 (0 B) duplicate files
```

And same story (report) for `fclones . --names '*.jpg'`

It looks like `fclones` does not see these files correctly. I thought this is because of their long names with whitespaces (sorry, these are names generated by my phone). I renamed two duplicates for simples names like `a.jpg` and `b.jpg` but I got same results - no duplications found.

Interestingly I tracked `fclones` by `strace` and there is no single strace log which claims `fclones` reads any files in this location.

Finally, I copied all these files to local directory on my disk and... same results - no duplications found.

Please let me know if you would need additional data about this issue to diagnose this problem

Thanks in advance


Discussion / hints:
To descend into directories, you need to add `-R` flag.
The correct command is `fclones -R .`
If you don't want recursive scan, but only analyze files in the current dir, run `fclones *`

Anyway you gave me an idea for a warning message. I can detect if inputs are directories and -R was not specified.
Actually I think I could get rid of `-R` option and just default to recursive scan. 
Most of the time I found recursive scanning is what users want.
If someone insists on scanning files only, there is `--depth`, so you could do `--depth 1` to match the default behaviour of `fdupes`. 

Fix patch:
diff --git a/Cargo.lock b/Cargo.lock
index a962159..fba3c71 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -324,7 +324,7 @@ checksum = "a357d28ed41a50f9c765dbfe56cbc04a64e53e5fc58ba79fbc34c10ef3df831f"
 
 [[package]]
 name = "fclones"
-version = "0.10.2"
+version = "0.11.0"
 dependencies = [
  "atomic-counter",
  "block-utils",
diff --git a/Cargo.toml b/Cargo.toml
index f398b54..4dde55b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "fclones"
-version = "0.10.2"
+version = "0.11.0"
 description = "Finds duplicate, unique, under- or over-replicated files"
 authors = ["Piotr Kołaczkowski <pkolaczk@gmail.com>"]
 repository = "https://github.com/pkolaczk/fclones"
diff --git a/README.md b/README.md
index fd553eb..e5b3bf9 100644
--- a/README.md
+++ b/README.md
@@ -74,60 +74,46 @@ are attached directly to [Releases](https://github.com/pkolaczk/fclones/releases
 The build will write the binary to `.cargo/bin/fclones`. 
 
 ## Usage
-Find duplicate files in the current directory, without descending into subdirectories 
-(in Unix-like shells with globbing support):
+Find duplicate, unique, under-replicated or over-replicated files in the current directory, 
+including subdirectories:
 
-    fclones * 
+    fclones .
+    fclones . --unique 
+    fclones . --rf-under 3
+    fclones . --rf-over 3
 
-Find duplicate files in the current directory, without descending into subdirectories (portable):
- 
-    fclones . -R --depth 1 
-
-Find common files in two directories, without descending into subdirectories 
-(in Unix-like shells with globbing support):
-
-    fclones dir1/* dir2/*  
-
-Find common files in two directories, without descending into subdirectories (portable):
-
-    fclones dir1 dir2 -R --depth 1  
-
-Find duplicate files in the current directory, including subdirectories:
-
-    fclones . -R
-    
-Find unique files in the current directory and its subdirectories:
-    
-    fclones . -R --unique 
+You can search in multiple directories:
 
-Find files that have more than 3 replicas:
+    fclones dir1 dir2 dir3
 
-    fclones . -R --rf-over 3
+Limiting recursion depth:
     
-Find files that have less than 3 replicas:
+    fclones . --depth 1   # scan only files in the current dir, skip subdirs
+    fclones * --depth 0   # similar as above in shells that expand `*` 
 
-    fclones . -R --rf-under 3
+Caution: Versions up to 0.10 did not descend into directories by default.
+In those old versions, add `-R` flag to enable recursive directory walking.
 
 ### Filtering
 Find duplicate files of size at least 100 MB: 
 
-    fclones . -R -s 100M
+    fclones . -s 100M
 
-Find duplicate pictures in the current directory:
+Filter by file name or path pattern:
 
-    fclones . -R --names '*.jpg' '*.png' 
+    fclones . --names '*.jpg' '*.png' 
                 
 Run `fclones` on files selected by `find` (note: this is likely slower than built-in filtering):
 
-    find . -name '*.c' | fclones --stdin
+    find . -name '*.c' | fclones --stdin --depth 0
 
 Follow symbolic links, but don't escape out of the home folder:
 
-    fclones . -R -L --paths '/home/**'
+    fclones . -L --paths '/home/**'
     
 Exclude a part of the directory tree from the scan:
 
-    fclones / -R --exclude '/dev/**' '/proc/**'    
+    fclones / --exclude '/dev/**' '/proc/**'    
     
 ### Preprocessing files
 Use `--transform` option to safely transform files by an external command.
@@ -135,25 +121,26 @@ By default, the transformation happens on a copy of file data, to avoid accident
 
 Strip exif before matching duplicate jpg images:
 
-    fclones . -R --names '*.jpg' --caseless --transform 'exiv2 -d a $IN' --in-place     
+    fclones . --names '*.jpg' --caseless --transform 'exiv2 -d a $IN' --in-place     
     
 ### Other    
     
 List more options:
     
-    fclones -h
+    fclones -h      # short help
+    fclones --help  # detailed help
 
 ### Notes on quoting and path globbing
 * On Unix-like systems, when using globs, one must be very careful to avoid accidental expansion of globs by the shell.
   In many cases having globs expanded by the shell instead of by `fclones` is not what you want. In such cases, you
   need to quote the globs:
     
-      fclones . -R --names '*.jpg'       
+      fclones . --names '*.jpg'       
        
 * On Windows, the default shell doesn't remove quotes before passing the arguments to the program, 
   therefore you need to pass globs unquoted:
   
-      fclones . -R --names *.jpg
+      fclones . --names *.jpg
       
 * On Windows, the default shell doesn't support path globbing, therefore wildcard characters such as * and ? used 
   in paths will be passed literally, and they are likely to create invalid paths. For example, the following 
@@ -162,9 +149,8 @@ List more options:
   
       fclones *
       
-  If you need path globbing, and your shell does not support it, 
-  use a combination of recursive search `-R` with `--depth` limit and 
-  built-in path globbing provided by `--names` or `--paths`.     
+  If you need path globbing, and your shell does not support it,
+  use the builtin path globbing provided by `--names` or `--paths`.     
                           
 ## The Algorithm
 Files are processed in several stages. Each stage except the last one is parallel, but 
diff --git a/src/config.rs b/src/config.rs
index b5ea3c2..1cdbc90 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -82,11 +82,11 @@ pub struct Config {
     #[structopt(short = "I", long)]
     pub stdin: bool,
 
-    /// Descends into directories recursively
-    #[structopt(short = "R", long)]
-    pub recursive: bool,
-
-    /// Limits recursion depth
+    /// Limits recursion depth.
+    ///
+    /// 0 disables descending into directories.
+    /// 1 descends into directories specified explicitly as input paths,
+    /// but does not descend into subdirectories.
     #[structopt(short = "d", long)]
     pub depth: Option<usize>,
 
@@ -208,7 +208,11 @@ pub struct Config {
     #[structopt(short = "Q", long)]
     pub quiet: bool,
 
-    /// A list of input paths. Accepts files and directories.
+    /// A list of input paths.
+    ///
+    /// Accepts files and directories.
+    /// By default descends into directories recursively, unless a recursion depth
+    /// limit is specified with `--depth`.
     #[structopt(parse(from_os_str), required_unless("stdin"))]
     pub paths: Vec<PathBuf>,
 }
diff --git a/src/lib.rs b/src/lib.rs
index 8aebbfb..e29f997 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -374,7 +374,6 @@ fn scan_files(ctx: &AppCtx<'_>) -> Vec<Vec<FileInfo>> {
     let max_size = config.max_size.unwrap_or(FileLen::MAX);
 
     let mut walk = Walk::new();
-    walk.recursive = config.recursive;
     walk.depth = config.depth.unwrap_or(usize::MAX);
     walk.skip_hidden = config.skip_hidden;
     walk.follow_links = config.follow_links;
@@ -836,7 +835,6 @@ fn group_by_contents(
 /// let log = Log::new();
 /// let mut config = Config::default();
 /// config.paths = vec![PathBuf::from("/path/to/a/dir")];
-/// config.recursive = true;
 ///
 /// let groups = group_files(&config, &log).unwrap();
 /// println!("Found {} groups: ", groups.len());
diff --git a/src/main.rs b/src/main.rs
index ad4d05e..d687bbf 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -80,6 +80,31 @@ fn main() {
     });
 
     let config: Config = Config::from_clap(&matches);
+
+    let mut access_error = false;
+    let mut has_input_files: bool = false;
+    for path in config.paths.iter() {
+        match std::fs::metadata(path) {
+            Ok(metadata) if metadata.is_dir() && config.depth == Some(0) => log.warn(format!(
+                "Skipping directory {} because recursive scan is disabled.",
+                path.display()
+            )),
+            Err(e) => {
+                log.err(format!("Can't access {}: {}", path.display(), e));
+                access_error = true;
+            }
+            Ok(_) => has_input_files = true,
+        }
+    }
+
+    if access_error {
+        exit(1);
+    }
+    if !has_input_files {
+        log.err("No input files");
+        exit(1);
+    }
+
     if config.quiet {
         log.no_progress = true;
     }
diff --git a/src/walk.rs b/src/walk.rs
index 241fd92..5a8df38 100644
--- a/src/walk.rs
+++ b/src/walk.rs
@@ -66,7 +66,6 @@ impl Entry {
 /// Many walks can be initiated from the same instance.
 pub struct Walk<'a> {
     pub base_dir: Arc<Path>,
-    pub recursive: bool,
     pub depth: usize,
     pub skip_hidden: bool,
     pub follow_links: bool,
@@ -87,7 +86,6 @@ impl<'a> Walk<'a> {
         let base_dir = Path::from(&current_dir().unwrap_or_default());
         Walk {
             base_dir: Arc::new(base_dir.clone()),
-            recursive: true,
             depth: usize::MAX,
             skip_hidden: false,
             follow_links: false,
@@ -219,7 +217,7 @@ impl<'a> Walk<'a> {
         F: Fn(Path) + Sync + Send,
         's: 'w,
     {
-        if self.recursive && level <= self.depth && self.path_selector.matches_dir(&path) {
+        if level < self.depth && self.path_selector.matches_dir(&path) {
             match std::fs::read_dir(path.to_path_buf()) {
                 Ok(rd) => {
                     for entry in Self::sorted_entries(path, rd) {


Test patch: