Repository: xonsh/xonsh

Issue:
EnvPath.add() doesn't maintain uniqueness with Path objects
## xonfig

<details>

```
+------------------+----------------------+
| xonsh            | 0.9.27               |
| Git SHA          | 71fe9014             |
| Commit Date      | Jan 29 08:58:58 2021 |
| Python           | 3.9.5                |
| PLY              | 3.11                 |
| have readline    | True                 |
| prompt toolkit   | None                 |
| shell type       | readline             |
| pygments         | 2.9.0                |
| on posix         | True                 |
| on linux         | True                 |
| distro           | ubuntu               |
| on darwin        | False                |
| on windows       | False                |
| on cygwin        | False                |
| on msys2         | False                |
| is superuser     | False                |
| default encoding | utf-8                |
| xonsh encoding   | utf-8                |
| encoding errors  | surrogateescape      |
| on jupyter       | False                |
| jupyter kernel   | None                 |
| xontrib 1        | apt_tabcomplete      |
| xontrib 2        | direnv               |
| xontrib 3        | kitty                |
| xontrib 4        | linuxbrew            |
+------------------+----------------------+
```

</details>

## Expected Behavior
`EnvPath.add(thing)` (e.g. `$PATH.add(thing)`) should not allow multiple copies of `thing` to enter the list, compared as strings.

## Current Behavior (and reproduction steps)
`EnvPath.add("str")` and `EnvPath.add(p"str")` do not detect each other's entries:

```
egnor@ostrich ~ $ $PATH
EnvPath(
['/usr/local/sbin',
 '/usr/local/bin',
 '/usr/sbin',
 '/usr/bin',
 '/sbin',
 '/bin',
 '/usr/games',
 '/usr/local/games',
 '/snap/bin']
)
egnor@ostrich ~ $ $PATH.add("/home/egnor/bin")
egnor@ostrich ~ $ $PATH.add(p"~/bin")
egnor@ostrich ~ $ $PATH
EnvPath(
['/usr/local/sbin',
 '/usr/local/bin',
 '/usr/sbin',
 '/usr/bin',
 '/sbin',
 '/bin',
 '/usr/games',
 '/usr/local/games',
 '/snap/bin',
 '/home/egnor/bin',
 '/home/egnor/bin']
)
```

This is because `EnvPath.add(data)` will put its data (of any type, `Path` or `str`) into the object directly, and `Path("foo") != "foo"`.

My first instinct was to have `EnvPath.add(data)` include `data = str(data)`, but then I realized that `EnvPath.__init__` was explicitly written to allow a mixed bag of `Path` and `str` objects, so I wasn't sure what was desired. `EnvPath` is documented as being a "list of strings" so I would expect it to stringify everything on entry, but maybe there's some concerns with that?

Note that `$PATH.add(p"~/bin")` seems like a very logical and idiomatic thing to put in one's `.xonshrc`, but due to this bug will end up with repeated copies when subshells occur.

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Hi!

> EnvPath.add(thing) (e.g. $PATH.add(thing)) should not allow multiple copies of thing to enter the list, compared as strings.

Why? 
AFAIK the OS doesn't mind and other shells allow this as well. 
The `.add` method is specifically documented not to add duplicates (with
flags controlling precise de-duping semantics), and in general when all the
items are "str" it does not, and that seems to be that method's whole
reason to exist, vs `.append` or whatnot.

The general use case is adding to `$PATH` or similar strings in one's
`.xonshrc` without ending up with an ever expanding path full of dupes due
to subshells that launch each other for various reasons.

This all works just fine but the `Path`/`str` mixture seems to break that.

>

Oh sorry I missed you used add instead of append

On Sat, Jul 17, 2021, 18:59 Daniel Egnor ***@***.***> wrote:

> The `.add` method is specifically documented not to add duplicates (with
> flags controlling precise de-duping semantics), and in general when all the
> items are "str" it does not, and that seems to be that method's whole
> reason to exist, vs `.append` or whatnot.
>
> The general use case is adding to `$PATH` or similar strings in one's
> `.xonshrc` without ending up with an ever expanding path full of dupes due
> to subshells that launch each other for various reasons.
>
> This all works just fine but the `Path`/`str` mixture seems to break that.
>
> >
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/xonsh/xonsh/issues/4366#issuecomment-881919089>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AELF3BIYPKBQ2DBQEPISGQDTYGSEVANCNFSM5AQJJPRA>
> .
>

Can confirm this is still a thing too. Also even with add used and replace it never removes all copies before inserting the one in front. So you will still have consistent issues.

![image](https://user-images.githubusercontent.com/1596188/132262503-ff91e3c7-aba0-4c73-baa9-964362bec31a.png)
![image](https://user-images.githubusercontent.com/1596188/132262525-70067811-40b9-4a71-9471-69e643824159.png)

Still think the suggestion from: https://github.com/xonsh/xonsh/issues/2562#issuecomment-357891119
Was sleekest way to clean this up after
![image](https://user-images.githubusercontent.com/1596188/132262586-98c3a08b-1bdd-43f4-8aca-017ac4644995.png)



Resolving patch:
diff --git a/xonsh/tools.py b/xonsh/tools.py
--- a/xonsh/tools.py
+++ b/xonsh/tools.py
@@ -280,10 +280,12 @@ def add(self, data, front=False, replace=False):
         None
 
         """
+        data = str(expand_path(data))
         if data not in self._l:
             self._l.insert(0 if front else len(self._l), data)
         elif replace:
-            self._l.remove(data)
+            # https://stackoverflow.com/a/25251306/1621381
+            self._l = list(filter(lambda x: x != data, self._l))
             self._l.insert(0 if front else len(self._l), data)
 
 
diff --git a/xonsh/xontribs_meta.py b/xonsh/xontribs_meta.py
--- a/xonsh/xontribs_meta.py
+++ b/xonsh/xontribs_meta.py
@@ -141,7 +141,7 @@ def define_xontribs():
             ),
         ),
         "avox_poetry": Xontrib(
-            url="github.com/jnoortheen/xontrib-avox-poetry",
+            url="https://github.com/jnoortheen/xontrib-avox-poetry",
             description="auto-activate venv as one cd into a poetry project folder. "
             "Activate ``.venv`` inside the project folder is also supported.",
             package=_XontribPkg(
@@ -193,7 +193,7 @@ def define_xontribs():
             package=core_pkg,
         ),
         "broot": Xontrib(
-            url="github.com/jnoortheen/xontrib-broot",
+            url="https://github.com/jnoortheen/xontrib-broot",
             description="supports broot with br alias",
             package=_XontribPkg(
                 name="xontrib-broot",
@@ -203,7 +203,7 @@ def define_xontribs():
             ),
         ),
         "powerline3": Xontrib(
-            url="github.com/jnoortheen/xontrib-powerline3",
+            url="https://github.com/jnoortheen/xontrib-powerline3",
             description="Powerline theme with native $PROMPT_FIELDS support.",
             package=_XontribPkg(
                 name="xontrib-powerline3",


Test patch:
diff --git a/tests/test_tools.py b/tests/test_tools.py
--- a/tests/test_tools.py
+++ b/tests/test_tools.py
@@ -942,6 +942,70 @@ def test_env_path_add(left, right, exp):
     assert exp == obs
 
 
+def test_env_path_add_replace_no_dupes_front_replace_existing():
+    # Test replaces without dupes when added to front when adding existing entry
+    path = EnvPath(
+        [os.pathsep.join(["home", "wakka"]), os.pathsep.join(["home", "wakka", "bin"])]
+    )
+    path.add(os.pathsep.join(["home", "wakka", "bin"]), front=True, replace=True)
+    assert path == [
+        os.pathsep.join(["home", "wakka", "bin"]),
+        os.pathsep.join(["home", "wakka"]),
+    ]
+
+
+def test_env_path_add_replace_no_dupes_front_replace_multiple():
+    # Test replaces without dupes when added to front when multiple existing occurrences
+    path = EnvPath(
+        [
+            os.pathsep.join(["home", "wakka"]),
+            os.pathsep.join(["home", "wakka", "bin"]),
+            os.pathsep.join(["home", "wakka", "bin"]),
+        ]
+    )
+    path.add(os.pathsep.join(["home", "wakka", "bin"]), front=True, replace=True)
+    assert path == [
+        os.pathsep.join(["home", "wakka", "bin"]),
+        os.pathsep.join(["home", "wakka"]),
+    ]
+
+
+def test_env_path_add_replace_no_dupes_back_replace_multiple():
+    # Test replaces without dupes when not added to front
+    path = EnvPath(
+        [
+            os.pathsep.join(["home", "wakka"]),
+            os.pathsep.join(["home", "wakka", "bin"]),
+            os.pathsep.join(["home", "wakka", "bin"]),
+        ]
+    )
+    path.add(os.pathsep.join(["home", "wakka", "bin"]), front=False, replace=True)
+    assert path == [
+        os.pathsep.join(["home", "wakka"]),
+        os.pathsep.join(["home", "wakka", "bin"]),
+    ]
+
+
+def test_env_path_add_pathlib():
+    os.pathsep.join(["home", "wakka", "bin"])
+    path = EnvPath(
+        [
+            os.pathsep.join(["home", "wakka"]),
+            os.pathsep.join(["home", "wakka", "bin"]),
+            os.pathsep.join(["home", "wakka", "bin"]),
+        ]
+    )
+    path.add(
+        pathlib.Path(os.pathsep.join(["home", "wakka", "bin"])),
+        front=False,
+        replace=True,
+    )
+    assert path == [
+        os.pathsep.join(["home", "wakka"]),
+        os.pathsep.join(["home", "wakka", "bin"]),
+    ]
+
+
 # helper
 def expand(path):
     return os.path.expanduser(os.path.expandvars(path))