Repository: sharkdp/fd (language: rust)

Issue:
Add `--ignore-file` flag
Basically identical to `rg`'s same flag.

The benefit of this is that sometimes you don't want to ignore every gitignore file found in the search path, but you do want to specifically ignore some files, so this flag comes in handy. Also, if I'm searching from `~`, I don't expect the gitignore file several levels down in some project to get picked up by default.


Discussion / hints:
Thank you for the feedback.

> Basically identical to rg's same flag.
> 
> The benefit of this is that sometimes you don't want to ignore every gitignore file found in the search path, but you do want to specifically ignore some files, so this flag comes in handy.

If I understand correctly, `rg`s flag is used to specify an *additional* ignore file (with low precedence), so this doesn't solve the use case you descibed(?).

Have you seen `fd`s `--exclude` option? It allows you to directly ignore a certain pattern:
``` bash
> fd --exclude '*.o' --exclude '/etc/secret-files' ...
```
Would this work for you?

> Also, if I'm searching from ~, I don't expect the gitignore file several levels down in some project to get picked up by default.

If you don't expect them to be picked up, you can use the `--no-ignore` option.
I think I may not have explained this clearly.

Say I have an ignore file at `~/.ignore` and another one at `~/(several levels down)/.gitignore`. My working directory is `~`.

If I run `fd --hidden --type directory .`, I expect the files in `~/.ignore` to be ignored, but not the ones in `~/(several levels down)/.gitignore`. But `fd` seems to pick up *every* ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least).

`--no-ignore` would work if I could then specify a specific ignore file, hence the request for the new flag ``--ignore-file`.
> But fd seems to pick up every ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least)

This is actually the behaviour I expect. If I am in a git repository I would expect fd to behave similar to `git ls-files | grep`. Usually, if I don't want to include files that are ignored by .gitignore files in their repository. 

As @sharkdp mentioned the --include-file option in rg doesn't behave the way you describe. It adds ignore rules _in addition to_ the ignore rules from .ignore and .gitignore files.
I'm not in a git repo though. Hmm, I'll test something when I get in to
work tomorrow
On Wed, Nov 1, 2017 at 9:49 PM Thayne McCombs <notifications@github.com>
wrote:

> But fd seems to pick up every ignore file in the tree rather than jus the
> top level one, which is unexpected behavior (for me at least)
>
> This is actually the behaviour I expect. If I am in a git repository I
> would expect fd to behave similar to git ls-files | grep. Usually, if I
> don't want to include files that are ignored by .gitignore files in their
> repository.
>
> As @sharkdp <https://github.com/sharkdp> mentioned the --include-file
> option in rg doesn't behave the way you describe. It adds ignore rules *in
> addition to* the ignore rules from .ignore and .gitignore files.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/sharkdp/fd/issues/156#issuecomment-341317021>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AH8KTOCyxGHGFCsOikyXt-MiLlMwdc9nks5syUn0gaJpZM4QMFAH>
> .
>
-- 
- Alok

My bad, its behavior is identical to `rg`. I still think the option to explicitly pass a set of ignore files and to only use those files for ignore patterns isn't a bad one, but I understand that's something extra.
I'd like to chime in in support of a command line option like this. `ag` and `rg` both honor _both_ `.gitignore` and a separate `.ignore` file. 

I'm currently using fzf to quick-find files in vim, supported by `rg --files` as my `DEFAULT_FZF_COMMAND`. Naturally, I have ripgrep set up to ignore all kinds of binary files (like images) that I would never want to open in vim, but which don't belong in `.gitignore`. 

While I certainly wouldn't want `fd` to honor my `.ignore` file all the time (or even by default), it would be nice to have the option to.
> I'd like to chime in in support of a command line option like this. ag and rg both honor both .gitignore and a separate .ignore file.

*fd* does that, too.

> While I certainly wouldn't want fd to honor my .ignore file all the time (or even by default), it would be nice to have the option to.

Okay, so you would really like to have the `--ignore-file` flag of ripgrep (with the exact same semantics)? In this case, let's re-open this.
> fd does that, too.

So it does! My mistake. I think the bigger issue then is simply that it's not mentioned in the man page or the README. 

> Okay, so you would really like to have the --ignore-file flag of ripgrep (with the exact same semantics)? In this case, let's re-open this.

Thanks; it'd definitely be nice to be able to rely on predefined ignore files to specify large sets of non-default ignore patterns.
After some more experimentation, it appears that `fd` also honors `.rgignore` by default (but not `.agignore` or `.fdignore`).

IMHO it would be saner to honor `.gitignore` and `.ignore` only, and leave `.<xx>ignore` files to apply to their respective utilities. If I want my `rgignore` rules to apply to `fd`, I'll either put them in `.ignore` or hardlink `.rgignore` and `.fdignore` to each other.

On a separate note, the manpage also contains an error (it says to use `--type s` for symlinks, where the actual option is `--type l`).

(If it would save you some trouble, I'd be happy to amend the documentation myself to clarify this point once I have a clear understand of how things work, and once they work the way you intend.)
> After some more experimentation, it appears that fd also honors .rgignore by default (but not .agignore or .fdignore).

This is due to our usage of the `ignore` crate, that is also used by ripgrep. This crate has both `.ignore` and `.rgignore` hardcoded:

https://github.com/BurntSushi/ripgrep/blob/c4e194538472de2cd74664a9a016b9c25c0e800b/ignore/src/dir.rs#L213

However, if my research is correct, the `.rgignore` files (as well as the `.agignore` files for *ag*) are deprecated and were superseded by `.ignore`. So `.rgignore` only exists for legacy reasons and should not be used anymore.

However, you have a fair point, since users might very well want to ignore different files for *fd* as compared to *rg*/*ag*. If `.ignore` is really meant for search-in-file tools, we should probably *not* respect these files and introduce `.fdignore` files instead?

> On a separate note, the manpage also contains an error (it says to use --type s for symlinks, where the actual option is --type l).

Thanks, fixed!

> (If it would save you some trouble, I'd be happy to amend the documentation myself to clarify this point once I have a clear understand of how things work, and once they work the way you intend.)

That would be great! Any contributions are always very much appreciated.
> However, if my research is correct, the `.rgignore` files (as well as the `.agignore` files for _ag_) are deprecated and were superseded by `.ignore`.

Yep, [this is definitely true](https://github.com/BurntSushi/ripgrep/issues/196#issuecomment-256695151). From @burntsushi himself:

> Please use `.ignore`. `.rgignore` is deprecated.

---

> If `.ignore` is really meant for search-in-file tools...

I can't say to a certainty what `.ignore` is for. AFAIK, it's a new convention that has been established by _ag_ and _rg._ In my opinion, it's not a very good one — there are many programs that can be configured with ignorefiles: not just _rg,_ _ag,_ and _fd,_ but git, syncthing, stow, and others. Why should a couple of popular grep utilities get an exemption on namespacing here? 

As to how _fd_ should handle this problem, I think the question should really be, _how much deviation from `find` can you justify in the name of sane defaults?_ IMO, ignoring hidden files and git-ignored files is reasonable, but lumping in _rg_ and _ag_ ignores is pushing it. And I think the remainder of use cases could be reasonably resolved by supporting `~/.fdignore` (ideally with an XDG-compatible alternative, like `~/.config/fdignore`), which, again, could be hardlinked to `ignore` if users really want them to be the same.

But you're the boss here, so what do you think?
I'd just like to chime in and say that I'd support simple changes to the `ignore` crate to iron out this UX, if y'all so choose.

Fix patch:
diff --git a/src/app.rs b/src/app.rs
index 1f823db6c..14291a31b 100644
--- a/src/app.rs
+++ b/src/app.rs
@@ -122,6 +122,14 @@ pub fn build_app() -> App<'static, 'static> {
                 .number_of_values(1)
                 .multiple(true),
         )
+        .arg(
+            arg("ignore-file")
+                .long("ignore-file")
+                .takes_value(true)
+                .value_name("path")
+                .number_of_values(1)
+                .multiple(true),
+        )
         .arg(
             arg("color")
                 .long("color")
@@ -220,6 +228,9 @@ fn usage() -> HashMap<&'static str, Help> {
         , "Exclude entries that match the given glob pattern"
         , "Exclude files/directories that match the given glob pattern. This overrides any \
            other ignore logic. Multiple exclude patterns can be specified.");
+    doc!(h, "ignore-file"
+        , "Add a custom ignore-file in .gitignore format"
+        , "Add a custom ignore-file in .gitignore format. These files have a low precedence.");
     doc!(h, "color"
         , "When to use colors: never, *auto*, always"
         , "Declare when to use color for the pattern match output:\n  \
diff --git a/src/internal.rs b/src/internal.rs
index 6b3a473e9..7b6ceebde 100644
--- a/src/internal.rs
+++ b/src/internal.rs
@@ -10,6 +10,7 @@ use std::ffi::OsString;
 use std::process;
 use std::time;
 use std::io::Write;
+use std::path::PathBuf;
 
 use exec::CommandTemplate;
 use lscolors::LsColors;
@@ -92,6 +93,9 @@ pub struct FdOptions {
 
     /// A list of glob patterns that should be excluded from the search.
     pub exclude_patterns: Vec<String>,
+
+    /// A list of custom ignore files.
+    pub ignore_files: Vec<PathBuf>,
 }
 
 /// Print error message to stderr and exit with status `1`.
diff --git a/src/main.rs b/src/main.rs
index 65815d065..a57b6d5ba 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -191,6 +191,10 @@ fn main() {
             .values_of("exclude")
             .map(|v| v.map(|p| String::from("!") + p).collect())
             .unwrap_or_else(|| vec![]),
+        ignore_files: matches
+            .values_of("ignore-file")
+            .map(|vs| vs.map(PathBuf::from).collect())
+            .unwrap_or_else(|| vec![]),
     };
 
     match RegexBuilder::new(&pattern_regex)
diff --git a/src/walk.rs b/src/walk.rs
index f151ea676..adf5bcd3d 100644
--- a/src/walk.rs
+++ b/src/walk.rs
@@ -14,6 +14,7 @@ use internal::{error, FdOptions, EXITCODE_SIGINT, MAX_BUFFER_LENGTH};
 use output;
 
 use std::process;
+use std::error::Error;
 use std::path::PathBuf;
 use std::sync::{Arc, Mutex};
 use std::sync::atomic::{AtomicBool, Ordering};
@@ -76,6 +77,22 @@ pub fn scan(path_vec: &[PathBuf], pattern: Arc<Regex>, config: Arc<FdOptions>) {
         walker.add_custom_ignore_filename(".fdignore");
     }
 
+    for ignore_file in &config.ignore_files {
+        let result = walker.add_ignore(ignore_file);
+        if let Some(err) = result {
+            match err {
+                ignore::Error::Partial(_) => (),
+                _ => {
+                    error(&format!(
+                        "Error while parsing custom ignore file '{}': {}.",
+                        ignore_file.to_string_lossy(),
+                        err.description()
+                    ));
+                }
+            }
+        }
+    }
+
     for path_entry in path_iter {
         walker.add(path_entry.as_path());
     }


Test patch:
diff --git a/tests/tests.rs b/tests/tests.rs
index fc720806f..a1790a20b 100644
--- a/tests/tests.rs
+++ b/tests/tests.rs
@@ -297,9 +297,9 @@ fn test_no_ignore() {
     );
 }
 
-/// Custom ignore files
+/// .gitignore and .fdignore
 #[test]
-fn test_custom_ignore() {
+fn test_gitignore_and_fdignore() {
     let files = &[
         "ignored-by-nothing",
         "ignored-by-fdignore",
@@ -335,7 +335,7 @@ fn test_custom_ignore() {
     );
 }
 
-/// Precedence of custom ignore files
+/// Precedence of .fdignore files
 #[test]
 fn test_custom_ignore_precedence() {
     let dirs = &["inner"];
@@ -378,6 +378,25 @@ fn test_no_ignore_vcs() {
     );
 }
 
+/// Custom ignore files (--ignore-file)
+#[test]
+fn test_custom_ignore_files() {
+    let te = TestEnv::new(DEFAULT_DIRS, DEFAULT_FILES);
+
+    // Ignore 'C.Foo2' and everything in 'three'.
+    fs::File::create(te.test_root().join("custom.ignore"))
+        .unwrap()
+        .write_all(b"C.Foo2\nthree")
+        .unwrap();
+
+    te.assert_output(
+        &["--ignore-file", "custom.ignore", "foo"],
+        "a.foo
+        one/b.foo
+        one/two/c.foo",
+    );
+}
+
 /// Ignored files with ripgrep aliases (-u / -uu)
 #[test]
 fn test_no_ignore_aliases() {