Repository: rust-lang/cargo (rust)

Issue:
shell_completions test issues
### Problem

The new shell_completions tests have a few issues that need to be resolved before they can be enabled.

1. The tests randomly fail. I have seen `fish` and `zsh` randomly fail.  https://github.com/rust-lang/cargo/pull/14541#issuecomment-2351096767 indicates that it might be an interactive timeout.
2. They require external tools to be installed to run. Tests generally should not require that.
    * Looking for the binaries is not sufficient, since for example the `bash` test also requires the bash-completions package to be installed.
3. The elvish test has a weird failure for me:
```
++++ actual:   In-memory
        1 + Deprecation: the legacy temporary assignment syntax is deprecated; use "tmp" instead
        2 +   /tmp/cargo/target/tmp/cit/t0/home/elvish/rc.elv:3:7: eval (E:CARGO_COMPLETE=elvish cargo | slurp)
```
4. Tests should be using the `ignore` attribute so that it reports in the output when it isn't running. Currently they exit with `return;` on macos, but that should use the `ignore` attribute instead.
5. Should figure out why the tests are ignored on macos, since I would expect that to work with homebrew. I'm unaware why these are currently ignored.

### Steps

1. `cargo test --test testsuite -- shell_completions`


### Possible Solution(s)

One idea I had for limiting where these tests run is to add a CI_EXTENDED environment variable. The tests would only be required to run if that environment variable is set, and then we can set that in the appropriate jobs (like `test`). Then the cargo_test macro could have something like `requires_extended_fish` or something like that. That would also be useful for the currently hard-coded `hg` and `lldb`. 

However, that doesn't help with some of the more complex issues like the bash-completions problem.

We could extend the cargo_test attribute to have `ignore_macos` to fix the macos ignore problem.

### Notes

_No response_

### Version

Currently on 643a025b3c3ad6f7d3acea558d223784ea8ab932 of master.



Pull request:
test: Remove completion tests
The tests are intended to spot check that shell completions are registered correctly.  That is a low change, low risk area.  For shell integration, we're relying on `clap_complete`s tests. For our own candidates, we should test the candidate generation directly, rather than end-to-end.

This reverts parts of commit e7ca9bec80ab5c010c1a84690816da7b64008257, reversing changes made to bd5f32bb1c7ca273b5d86815bf0ae4adba59ddd8.

Fixes #14545

<!-- homu-ignore:start -->
<!--
Thanks for submitting a pull request 🎉! Here are some tips for you:

* If this is your first contribution, read "Cargo Contribution Guide" first:
  https://doc.crates.io/contrib/
* Run `cargo fmt --all` to format your code changes.
* Small commits and pull requests are always preferable and easy to review.
* If your idea is large and needs feedback from the community, read how:
  https://doc.crates.io/contrib/process/#working-on-large-features
* Cargo takes care of compatibility. Read our design principles:
  https://doc.crates.io/contrib/design.html
* When changing help text of cargo commands, follow the steps to generate docs:
  https://github.com/rust-lang/cargo/tree/master/src/doc#building-the-man-pages
* If your PR is not finished, set it as "draft" PR or add "WIP" in its title.
* It's ok to use the CI resources to test your PR, but please don't abuse them.

### What does this PR try to resolve?

Explain the motivation behind this change.
A clear overview along with an in-depth explanation are helpful.

You can use `Fixes #<issue number>` to associate this PR to an existing issue.

### How should we test and review this PR?

Demonstrate how you test this change and guide reviewers through your PR.
With a smooth review process, a pull request usually gets reviewed quicker.

If you don't know how to write and run your tests, please read the guide:
https://doc.crates.io/contrib/tests

### Additional information

Other information you want to mention in this PR, such as prior arts,
future extensions, an unresolved problem, or a TODO list.
-->
<!-- homu-ignore:end -->


Patch:
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 478b164d8..893c87124 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -173,13 +173,6 @@ jobs:
     - run: rustup target add ${{ matrix.other }}
     - run: rustup component add rustc-dev llvm-tools-preview rust-docs
       if: startsWith(matrix.rust, 'nightly')
-    # Install fish, zsh, and elvish only on Ubuntu systems
-    - name: Install fish, zsh, and elvish on Ubuntu
-      run: sudo apt update -y && sudo apt install fish zsh elvish -y
-      if: matrix.os == 'ubuntu-latest'
-    - name: Install fish, elvish on macOS
-      run: brew install fish elvish
-      if: matrix.os == 'macos-14' || matrix.os == 'macos-13'
     - run: sudo apt update -y && sudo apt install lldb gcc-multilib libsecret-1-0 libsecret-1-dev -y
       if: matrix.os == 'ubuntu-latest'
     - run: rustup component add rustfmt || echo "rustfmt not available"
@@ -234,7 +227,6 @@ jobs:
       - run: rustup update --no-self-update stable && rustup default stable
       - run: rustup target add i686-unknown-linux-gnu
       - run: sudo apt update -y && sudo apt install gcc-multilib libsecret-1-0 libsecret-1-dev -y
-      - run: sudo apt update -y && sudo apt install fish zsh elvish -y
       - run: rustup component add rustfmt || echo "rustfmt not available"
       - run: cargo test -p cargo
         env:
diff --git a/Cargo.lock b/Cargo.lock
index 6a9cd6437..dea2efe0e 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -134,12 +134,6 @@ version = "1.7.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "69f7f8c3906b62b754cd5326047894316021dcfe5a194c8ea52bdd94934a3457"
 
-[[package]]
-name = "arrayvec"
-version = "0.7.6"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
-
 [[package]]
 name = "autocfg"
 version = "1.3.0"
@@ -291,7 +285,6 @@ dependencies = [
  "clap",
  "clap_complete",
  "color-print",
- "completest-pty",
  "crates-io",
  "curl",
  "curl-sys",
@@ -618,23 +611,6 @@ version = "1.0.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "d3fd119d74b830634cea2a0f58bbd0d54540518a14397557951e79340abc28c0"
 
-[[package]]
-name = "completest"
-version = "0.4.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "410de1ffe61368aa040f599747584e9e3d19235cf4045be6159edb167ef45ddb"
-
-[[package]]
-name = "completest-pty"
-version = "0.5.5"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "fbd2f22a999db122bd2861c504aa363bbacaa32ebea29edf6924ee6cfe044313"
-dependencies = [
- "completest",
- "ptyprocess",
- "vt100",
-]
-
 [[package]]
 name = "const-oid"
 version = "0.9.6"
@@ -2351,15 +2327,6 @@ dependencies = [
  "libc",
 ]
 
-[[package]]
-name = "memoffset"
-version = "0.7.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5de893c32cde5f383baa4c04c5d6dbdd735cfd4a794b0debdb2bb1b421da5ff4"
-dependencies = [
- "autocfg",
-]
-
 [[package]]
 name = "minimal-lexical"
 version = "0.2.1"
@@ -2384,19 +2351,6 @@ dependencies = [
  "windows-sys 0.48.0",
 ]
 
-[[package]]
-name = "nix"
-version = "0.26.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "598beaf3cc6fdd9a5dfb1630c2800c7acd31df7aaf0f565796fba2b53ca1af1b"
-dependencies = [
- "bitflags 1.3.2",
- "cfg-if",
- "libc",
- "memoffset",
- "pin-utils",
-]
-
 [[package]]
 name = "nom"
 version = "7.1.3"
@@ -2719,12 +2673,6 @@ version = "0.2.14"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "bda66fc9667c18cb2758a2ac84d1167245054bcf85d5d1aaa6923f45801bdd02"
 
-[[package]]
-name = "pin-utils"
-version = "0.1.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
-
 [[package]]
 name = "pkcs8"
 version = "0.10.2"
@@ -2828,15 +2776,6 @@ dependencies = [
  "unarray",
 ]
 
-[[package]]
-name = "ptyprocess"
-version = "0.4.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "7e05aef7befb11a210468a2d77d978dde2c6381a0381e33beb575e91f57fe8cf"
-dependencies = [
- "nix",
-]
-
 [[package]]
 name = "pulldown-cmark"
 version = "0.11.0"
@@ -3855,39 +3794,6 @@ version = "0.9.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "49874b5167b65d7193b8aba1567f5c7d93d001cafc34600cee003eda787e483f"
 
-[[package]]
-name = "vt100"
-version = "0.15.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "84cd863bf0db7e392ba3bd04994be3473491b31e66340672af5d11943c6274de"
-dependencies = [
- "itoa 1.0.11",
- "log",
- "unicode-width",
- "vte",
-]
-
-[[package]]
-name = "vte"
-version = "0.11.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f5022b5fbf9407086c180e9557be968742d839e68346af7792b8592489732197"
-dependencies = [
- "arrayvec",
- "utf8parse",
- "vte_generate_state_changes",
-]
-
-[[package]]
-name = "vte_generate_state_changes"
-version = "0.1.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2e369bee1b05d510a7b4ed645f5faa90619e05437111783ea5848f28d97d3c2e"
-dependencies = [
- "proc-macro2",
- "quote",
-]
-
 [[package]]
 name = "wait-timeout"
 version = "0.2.0"
diff --git a/Cargo.toml b/Cargo.toml
index a1044b553..0b0782099 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -38,7 +38,6 @@ cargo_metadata = "0.18.1"
 clap = "4.5.18"
 clap_complete = { version = "4.5.29", features = ["unstable-dynamic"] }
 color-print = "0.3.6"
-completest-pty = "0.5.3"
 core-foundation = { version = "0.10.0", features = ["mac_os_10_7_support"] }
 crates-io = { version = "0.40.4", path = "crates/crates-io" }
 criterion = { version = "0.5.1", features = ["html_reports"] }
@@ -242,7 +241,6 @@ features = [
 [dev-dependencies]
 annotate-snippets = { workspace = true, features = ["testing-colors"] }
 cargo-test-support.workspace = true
-completest-pty.workspace = true
 gix = { workspace = true, features = ["revision"] }
 same-file.workspace = true
 snapbox.workspace = true


Test patch:
diff --git a/tests/testsuite/main.rs b/tests/testsuite/main.rs
index a5bd89a8c..0c0434271 100644
--- a/tests/testsuite/main.rs
+++ b/tests/testsuite/main.rs
@@ -166,7 +166,6 @@ mod rustflags;
 mod rustup;
 mod script;
 mod search;
-mod shell_completions;
 mod shell_quoting;
 mod source_replacement;
 mod ssh;
diff --git a/tests/testsuite/shell_completions.rs b/tests/testsuite/shell_completions.rs
deleted file mode 100644
index 2b96e2667..000000000
--- a/tests/testsuite/shell_completions.rs
+++ /dev/null
@@ -1,204 +0,0 @@
-#![cfg(unix)]
-
-use cargo_test_support::cargo_test;
-use completest_pty::Runtime;
-use snapbox::assert_data_eq;
-
-#[cargo_test]
-#[ignore = "disabled, see https://github.com/rust-lang/cargo/issues/14545"]
-fn bash() {
-    // HACK: At least on CI, bash is not working on macOS
-    if cfg!(target_os = "macos") {
-        return;
-    }
-
-    let input = "cargo \t\t";
-    let expected = snapbox::str![
-        "% 
---version          --help             check              install            read-manifest      update
---list             -V                 clean              locate-project     remove             vendor
---explain          -v                 config             login              report             verify-project
---verbose          -q                 doc                logout             run                version
---quiet            -C                 fetch              metadata           rustc              yank
---color            -Z                 fix                new                rustdoc            
---locked           -h                 generate-lockfile  owner              search             
---offline          add                help               package            test               
---frozen           bench              info               pkgid              tree               
---config           build              init               publish            uninstall          "
-    ];
-    let actual = complete(input, "bash");
-    assert_data_eq!(actual, expected);
-}
-
-#[cargo_test]
-#[ignore = "disabled, see https://github.com/rust-lang/cargo/issues/14545"]
-fn elvish() {
-    // HACK: At least on CI, elvish is not working on macOS
-    if cfg!(target_os = "macos") {
-        return;
-    }
-
-    let input = "cargo \t\t";
-    let expected = snapbox::str![
-        "% cargo --config
- COMPLETING argument  
---color    --version  check              install         read-manifest  update        
---config   -C         clean              locate-project  remove         vendor        
---explain  -V         config             login           report         verify-project
---frozen   -Z         doc                logout          run            version       
---help     -h         fetch              metadata        rustc          yank          
---list     -q         fix                new             rustdoc      
---locked   -v         generate-lockfile  owner           search       
---offline  add        help               package         test         
---quiet    bench      info               pkgid           tree         
---verbose  build      init               publish         uninstall    "
-    ];
-    let actual = complete(input, "elvish");
-    assert_data_eq!(actual, expected);
-}
-
-#[cargo_test]
-#[ignore = "disabled, see https://github.com/rust-lang/cargo/issues/14545"]
-fn fish() {
-    // HACK: At least on CI, fish is not working on macOS
-    if cfg!(target_os = "macos") {
-        return;
-    }
-
-    let input = "cargo \t\t";
-    let expected = snapbox::str![
-        "% cargo 
---version                                                                                  (Print version info and exit)
---list                                                                                         (List installed commands)
---explain                                                      (Provide a detailed explanation of a rustc error message)
---verbose                                                        (Use verbose output (-vv very verbose/build.rs output))
---quiet                                                                                (Do not print cargo log messages)
---color                                                                                  (Coloring: auto, always, never)
---locked                                                                (Assert that `Cargo.lock` will remain unchanged)
---offline                                                                            (Run without accessing the network)
---frozen                                                          (Equivalent to specifying both --locked and --offline)
---config                                                                                (Override a configuration value)
---help                                                                                                      (Print help)
--V                                                                                         (Print version info and exit)
--v                                                               (Use verbose output (-vv very verbose/build.rs output))
--q                                                                                     (Do not print cargo log messages)
--C                                                            (Change to DIRECTORY before doing anything (nightly-only))
--Z                                             (Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details)
--h                                                                                                          (Print help)
-add                                                                     (Add dependencies to a Cargo.toml manifest file)
-bench                                                                        (Execute all benchmarks of a local package)
-build                                                              (Compile a local package and all of its dependencies)
-check                                                     (Check a local package and all of its dependencies for errors)
-clean                                                            (Remove artifacts that cargo has generated in the past)
-config                                                                                    (Inspect configuration values)
-doc                                                                                    (Build a package's documentation)
-fetch                                                                 (Fetch dependencies of a package from the network)
-fix                                                                  (Automatically fix lint warnings reported by rustc)
-generate-lockfile                                                                  (Generate the lockfile for a package)
-help                                                                              (Displays help for a cargo subcommand)
-info                                                               (Display information about a package in the registry)
-init                                                               (Create a new cargo package in an existing directory)
-install                                                                                          (Install a Rust binary)
-locate-project                                             (Print a JSON representation of a Cargo.toml file's location)
-login                                                                                            (Log in to a registry.)
-logout                                                                   (Remove an API token from the registry locally)
-metadata  (Output the resolved dependencies of a package, the concrete used versions including overrides, in machine-r…)
-new                                                                               (Create a new cargo package at <path>)
-owner                                                                     (Manage the owners of a crate on the registry)
-package                                                        (Assemble the local package into a distributable tarball)
-pkgid                                                                    (Print a fully qualified package specification)
-publish                                                                               (Upload a package to the registry)
-read-manifest                                                    (Print a JSON representation of a Cargo.toml manifest.)
-remove                                                             (Remove dependencies from a Cargo.toml manifest file)
-report                                                                   (Generate and display various kinds of reports)
-run                                                                       (Run a binary or example of the local package)
-rustc                                                        (Compile a package, and pass extra options to the compiler)
-rustdoc                                                 (Build a package's documentation, using specified custom flags.)
-search                                                  (Search packages in the registry. Default registry is crates.io)
-test                                      (Execute all unit and integration tests and build examples of a local package)
-tree                                                                (Display a tree visualization of a dependency graph)
-uninstall                                                                                         (Remove a Rust binary)
-update                                                          (Update dependencies as recorded in the local lock file)
-vendor                                                                   (Vendor all dependencies for a project locally)
-verify-project                                                                     (Check correctness of crate manifest)
-version                                                                                       (Show version information)
-yank                                                                              (Remove a pushed crate from the index)"];
-
-    let actual = complete(input, "fish");
-    assert_data_eq!(actual, expected);
-}
-
-#[cargo_test]
-#[ignore = "disabled, see https://github.com/rust-lang/cargo/issues/14545"]
-fn zsh() {
-    let input = "cargo \t\t";
-    let expected = snapbox::str![
-        "% cargo
---color            --version          check              install            read-manifest      update
---config           -C                 clean              locate-project     remove             vendor
---explain          -V                 config             login              report             verify-project
---frozen           -Z                 doc                logout             run                version
---help             -h                 fetch              metadata           rustc              yank
---list             -q                 fix                new                rustdoc            
---locked           -v                 generate-lockfile  owner              search             
---offline          add                help               package            test               
---quiet            bench              info               pkgid              tree               
---verbose          build              init               publish            uninstall          "
-    ];
-    let actual = complete(input, "zsh");
-    assert_data_eq!(actual, expected);
-}
-
-fn complete(input: &str, shell: &str) -> String {
-    let shell = shell.into();
-
-    // Load the runtime
-    let mut runtime = match shell {
-        "bash" => load_runtime::<completest_pty::BashRuntimeBuilder>("bash"),
-        "elvish" => load_runtime::<completest_pty::ElvishRuntimeBuilder>("elvish"),
-        "fish" => load_runtime::<completest_pty::FishRuntimeBuilder>("fish"),
-        "zsh" => load_runtime::<completest_pty::ZshRuntimeBuilder>("zsh"),
-        _ => panic!("Unsupported shell: {}", shell),
-    };
-
-    // Exec the completion
-    let term = completest_pty::Term::new();
-    let actual = runtime.complete(input, &term).unwrap();
-
-    actual
-}
-
-// Return the scratch directory to keep it not being dropped
-fn load_runtime<R: completest_pty::RuntimeBuilder>(shell: &str) -> Box<dyn completest_pty::Runtime>
-where
-    <R as completest_pty::RuntimeBuilder>::Runtime: 'static,
-{
-    let home = cargo_test_support::paths::home();
-
-    let bin_path = cargo_test_support::cargo_exe();
-    let bin_root = bin_path.parent().unwrap().to_owned();
-
-    let mut runtime = Box::new(R::new(bin_root, home).unwrap());
-
-    match shell {
-        "bash" => runtime
-            .register("", "source <(CARGO_COMPLETE=bash cargo)")
-            .unwrap(),
-        "elvish" => runtime
-            .register("", "eval (E:CARGO_COMPLETE=elvish cargo | slurp)")
-            .unwrap(),
-        "fish" => runtime
-            .register("cargo", "source (CARGO_COMPLETE=fish cargo | psub)")
-            .unwrap(),
-        "zsh" => runtime
-            .register(
-                "cargo",
-                "#compdef cargo
-source <(CARGO_COMPLETE=zsh cargo)",
-            )
-            .unwrap(),
-        _ => panic!("Unsupported shell: {}", shell),
-    }
-
-    runtime
-}