Repository: arrayfire/arrayfire-rust (language: rust)

Issue:
[Build] Build scripts silently ignores non-accessible DLLs
Description
===========
<!--
A short one or two line description of the error
-->

When looking for available DLLs, the build scripts treats all access errors like files that don't exist:

https://github.com/arrayfire/arrayfire-rust/blob/7cfde6b827f2976767f4e5dc6b8913fe3ef0140e/build.rs#L71-L76

If the current user has no read permissions on the `af.dll`, it is not added to the `backends` list:

https://github.com/arrayfire/arrayfire-rust/blob/7cfde6b827f2976767f4e5dc6b8913fe3ef0140e/build.rs#L419-L424

As a result, no arrayfire library is passed to the linker, which then throws a lot of errors because for the various `af_*` functions (see below).

The arrayfire build script should really trigger an error or panic if no backends are found instead of letting the users run into linker errors. Ideally, the error message would also mention that some files were found, but were not be accessible.

Build Environment
-----------------
Compiler version: rustc 1.57.0 (f1edd0429 2021-11-29)
Operating system: Windows 10
Build environment: (described above)

Error Log
---------
<!-- Output of the error log. -->

Lots of errors like these:
```
error LNK2019: unresolved external symbol af_retain_array referenced in function _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h3e2e43e5f6399194E
```



Discussion / hints:
> If the current user has no read permissions on the af.dll, it is not added to the backends list:

I don't see a problem in this scenario. If the current user doesn't have permissions, that should be fixed outside of rust's context. If a file can't be read by an user, it is essentially a non-existent file to that user. There is no way to know what are it's permissions. At least, I am not aware of any such mechanism. If you are aware of something like that, please do share and I shall look into it. You are also welcome to raise a PR with the necessary fix if that is something you are interested in.
The main problem is that there is no error or warning if no backends are found at all, i.e. when the `backends` list is empty. There is no way the linking can succeed in this case, so I think it makes sense to treat this case as an error in my opinion.

It's true that non-readable files cannot be used and must be fixed manually. However, if the user explicitly points the `AF_PATH` to such a directory, I don't think that silently ignoring these access errors is a good idea, as it gives the impression that the `AF_PATH` is valid and working.

So my proposal is: If no backends are found, we throw an error from the build script (it would result in an linker error later anyway). We also keep track of the I/O error that occurs for the main `af.dll`/`af.so` and display it as an additional help message. For example:

```
Error: No arrayfire backends found

help: - searched paths: ...
      - AF_PATH set to ....
      - a file named `af.dll` exist at ..., but an error occured while accessing it: ...
```
> The main problem is that there is no error or warning if no backends are found at all, i.e. when the backends list is empty. There is no way the linking can succeed in this case, so I think it makes sense to treat this case as an error in my opinion.

I wonder when such a scenario would happen unless it is a broken installation or broken build 🤔 Nevertheless, I agree an early break while build script is running might help parsing a larger linker error message. Feel free to raise a PR for the same if you can, Otherwise I shall look into it as soon as I can.

> It's true that non-readable files cannot be used and must be fixed manually. However, if the user explicitly points the AF_PATH to such a directory, I don't think that silently ignoring these access errors is a good idea, as it gives the impression that the AF_PATH is valid and working.
> ...
>  a file named `af.dll` exist at ..., but an error occured while accessing it: ...

How do you propose accessing the access of a backend(library file) ? Other than checking for it's existence, any other solution will end being similar to using the library file like the rust create does.
> I wonder when such a scenario would happen unless it is a broken installation or broken build

It was a non-standard installation in our case that was indeed broken :smile:. However, debugging it was much more difficult because the build script succeeded without errors. I'll create a PR to add an error message for that case.

> How do you propose accessing the access of a backend(library file) ?

We can check the [error kind of the `io::Error`](https://doc.rust-lang.org/stable/std/io/struct.Error.html#method.kind), which is [`NotFound`](https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.NotFound) if the file doesn't exist and [`PermissionDenied`](https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.PermissionDenied) if the permissions are wrong. At least it was like that in our case. 

Fix patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 967b0169..4e27f9a6 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -73,6 +73,8 @@ jobs:
     clippy:
         name: Clippy Lints
         runs-on: ubuntu-18.04
+        env:
+            AF_VER: 3.8.0
         steps:
             - name: Checkout Repository
               uses: actions/checkout@master
@@ -83,7 +85,29 @@ jobs:
                   toolchain: stable
                   override: true
                   components: clippy
+
+            - name: Cache ArrayFire
+              uses: actions/cache@v1
+              id: arrayfire
+              with:
+                  path: afbin
+                  key: ${{ runner.os }}-af-${{ env.AF_VER }}
+
+            - name: Download ArrayFire
+              # Only download and cache arrayfire if already not found
+              if: steps.arrayfire.outputs.cache-hit != 'true'
+              run: |
+                  wget --quiet http://arrayfire.s3.amazonaws.com/${AF_VER}/ArrayFire-v${AF_VER}_Linux_x86_64.sh
+                  chmod +x ./ArrayFire-v${AF_VER}_Linux_x86_64.sh
+                  mkdir afbin
+                  ./ArrayFire-v${AF_VER}_Linux_x86_64.sh --skip-license --exclude-subdir --prefix=./afbin
+                  rm ./afbin/lib64/libcu*.so*
+                  rm ./afbin/lib64/libafcuda*.so*
+                  rm ./ArrayFire-v${AF_VER}_Linux_x86_64.sh
+
             - name: Run clippy tool
+              env:
+                AF_PATH: ${{ github.workspace }}/afbin
               uses: actions-rs/cargo@v1
               with:
                   command: clippy
diff --git a/build.rs b/build.rs
index 5b55757a..c1fca518 100644
--- a/build.rs
+++ b/build.rs
@@ -58,21 +58,32 @@ struct Config {
     win_vs_toolset: String,
 }
 
-fn fail(s: &str) -> ! {
-    panic!("\n{}\n\nbuild script failed, must exit now", s)
+fn fail(msg: &str) -> ! {
+    eprintln!("ERROR: {}", msg);
+    std::process::exit(1);
 }
 
 fn dir_exists(location: &str) -> bool {
     match fs::metadata(location) {
         Ok(f) => f.is_dir(),
-        Err(_) => false,
+        Err(err) => {
+            if err.kind() != ErrorKind::NotFound {
+                eprintln!("WARNING: failed to access `{}`: {}", location, err);
+            }
+            false
+        }
     }
 }
 
 fn file_exists(location: &str) -> bool {
     match fs::metadata(location) {
         Ok(f) => f.is_file(),
-        Err(_) => false,
+        Err(err) => {
+            if err.kind() != ErrorKind::NotFound {
+                eprintln!("WARNING: failed to access `{}`: {}", location, err);
+            }
+            false
+        }
     }
 }
 
@@ -291,13 +302,9 @@ fn blob_backends(conf: &Config, build_dir: &std::path::Path) -> (Vec<String>, Ve
         let afpath = match env::var("AF_PATH") {
             Ok(af_path) => PathBuf::from(&af_path),
             Err(_) => {
-                println!(
-                    "WARNING! USE_LIB is defined,
-                          but AF_PATH is not found,"
-                );
-                println!(
-                    "Trying to find libraries from
-                          known default locations"
+                eprintln!(
+                    "WARNING: USE_LIB is defined, but AF_PATH is not found. Trying to find \
+                    libraries from known default locations."
                 );
                 if cfg!(target_os = "windows") {
                     PathBuf::from("C:\\Program Files\\ArrayFire\\v3\\")
@@ -449,6 +456,11 @@ fn main() {
     }
 
     let (backends, backend_dirs) = blob_backends(&conf, &build_dir);
+
+    if backends.is_empty() {
+        fail("no arrayfire backends found");
+    }
+
     for backend in backends.iter() {
         println!("cargo:rustc-link-lib=dylib={}", backend);
     }


Test patch: