Repository: ericsson/codechecker
Issue: CodeChecker cannot execute if there isn't a `clang` binary on the system

**Describe the bug**
If `clang` is not found in `PATH` anywhere at all, the `analyze` and `check` commands fail to go because they try to **always** read a Clang version.

**CodeChecker version**

(Note that even running something as simple and completely orthogonal as a package version check wants to access Clang!!!)

~~~~
╰─ ~/CodeChecker/build/CodeChecker/bin/CodeChecker version
CodeChecker analyzer version:
[WARNING 2021-01-13 17:56] - 'clang' binary can not be found in your PATH!
---------------------------------------------------------------
Kind                 | Version
---------------------------------------------------------------
Base package version | 6.16.0
Package build date   | 2021-01-13T17:50
Git commit ID (hash) | 9d23b7ba6d4c149a6fabf21579f08fc60ef4df1d
Git tag information  | 6.16
---------------------------------------------------------------

CodeChecker web version:
------------------------------------------------------------------------
Kind                          | Version
------------------------------------------------------------------------
Base package version          | 6.16.0
Package build date            | 2021-01-13T17:50
Git commit ID (hash)          | 9d23b7ba6d4c149a6fabf21579f08fc60ef4df1d
Git tag information           | 6.16
Server supported API (Thrift) | 6.37
Client API (Thrift)           | 6.37
------------------------------------------------------------------------
~~~~

**To Reproduce**
Steps to reproduce the behaviour:

1. Download and build CodeChecker.
2. Install `clang-tidy` from the package manager, or have a binary release of Clang-Tidy, or compile it locally. Either way, do **not** have `clang` in the `PATH`, only `clang-tidy`.
3. Try to run an `check --analyzers clang-tidy ...` or `analyze --analyzers clang-tidy`.
4. Observe error:

~~~~
╰─ which clang
clang not found

╰─ which clang-tidy
~/LLVM/Build/bin/clang-tidy

╰─ ~/CodeChecker/build/CodeChecker/bin/CodeChecker check -l compile_commands.json --analyzers clang-tidy
'clang' binary can not be found in your PATH!
'clang' binary can not be found in your PATH!
[WARNING 2021-01-13 17:56] - 'clang' binary can not be found in your PATH!
[ERROR 2021-01-13 17:56] - Running check failed.
Traceback (most recent call last):
  File "~/CodeChecker/build/CodeChecker/cc_bin/../lib/python3/codechecker_analyzer/cmd/check.py", line 819, in main
    analysis_exit_status = analyze_module.main(analyze_args)
  File "~/CodeChecker/build/CodeChecker/lib/python3/codechecker_analyzer/cmd/analyze.py", line 937, in main
    analyzer_env)
  File "~/CodeChecker/build/CodeChecker/lib/python3/codechecker_analyzer/analyzers/clangsa/version.py", line 73, in get
    errors="ignore")
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1278, in _execute_child
    executable = os.fsencode(executable)
  File "/usr/lib/python3.6/os.py", line 800, in fsencode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "~/CodeChecker/build/CodeChecker/cc_bin/../lib/python3/codechecker_analyzer/cmd/check.py", line 819, in main
    analysis_exit_status = analyze_module.main(analyze_args)
  File "~/CodeChecker/build/CodeChecker/lib/python3/codechecker_analyzer/cmd/analyze.py", line 937, in main
    analyzer_env)
  File "~/CodeChecker/build/CodeChecker/lib/python3/codechecker_analyzer/analyzers/clangsa/version.py", line 73, in get
    errors="ignore")
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1278, in _execute_child
    executable = os.fsencode(executable)
  File "/usr/lib/python3.6/os.py", line 800, in fsencode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "~/CodeChecker/build/CodeChecker/cc_bin/CodeChecker.py", line 141, in main
    sys.exit(args.func(args))
  File "~/CodeChecker/build/CodeChecker/cc_bin/../lib/python3/codechecker_analyzer/cmd/check.py", line 850, in main
    return analysis_exit_status
UnboundLocalError: local variable 'analysis_exit_status' referenced before assignment
~~~~

**Expected behaviour**
CodeChecker acknowledges the fact that Clang does not exist on the system and goes on with only running Clang-Tidy.

**Desktop (please complete the following information)**
 - OS: Ubuntu 18.04.5 LTS

**Additional context**
This issue, together with #2670 makes it really hard to use CodeChecker for actively testing Tidy (and Tidy-only) contributions. I do not wish to have Clang SA built on the system because it's a lot of effort to always rebuild features that are not being used.

The culprit in the source code is here:

https://github.com/Ericsson/codechecker/blob/9d23b7ba6d4c149a6fabf21579f08fc60ef4df1d/analyzer/codechecker_analyzer/cmd/analyze.py#L933-L937

Changed files:
analyzer/codechecker_analyzer/analyzer_context.py, analyzer/codechecker_analyzer/analyzer.py, analyzer/codechecker_analyzer/analyzers/analyzer_types.py, analyzer/codechecker_analyzer/cmd/analyze.py, analyzer/codechecker_analyzer/cmd/checkers.py

diff --git a/analyzer/codechecker_analyzer/analyzer.py b/analyzer/codechecker_analyzer/analyzer.py
index 54d744da..71b9a3c9 100644
--- a/analyzer/codechecker_analyzer/analyzer.py
+++ b/analyzer/codechecker_analyzer/analyzer.py
@@ -176,8 +176,9 @@ def perform_analysis(args, skip_handler, context, actions, metadata_tool,
 
     analyzers = args.analyzers if 'analyzers' in args \\
         else analyzer_types.supported_analyzers
-    analyzers, _ = analyzer_types.check_supported_analyzers(
+    analyzers, errored = analyzer_types.check_supported_analyzers(
         analyzers, context)
+    analyzer_types.check_available_analyzers(analyzers, errored)
 
     ctu_collect = False
     ctu_analyze = False
@@ -366,6 +367,8 @@ def perform_analysis(args, skip_handler, context, actions, metadata_tool,
     end_time = time.time()
     LOG.info("Analysis length: %s sec.", end_time - start_time)
 
+    analyzer_types.print_unsupported_analyzers(errored)
+
     metadata_tool['timestamps'] = {'begin': start_time,
                                    'end': end_time}
 
diff --git a/analyzer/codechecker_analyzer/analyzer_context.py b/analyzer/codechecker_analyzer/analyzer_context.py
index db7540c0..6f0f2c3f 100644
--- a/analyzer/codechecker_analyzer/analyzer_context.py
+++ b/analyzer/codechecker_analyzer/analyzer_context.py
@@ -339,8 +339,8 @@ class Context(object):
                 env_path = analyzer_env['PATH'] if analyzer_env else None
                 compiler_binary = find_executable(value, env_path)
                 if not compiler_binary:
-                    LOG.warning("'%s' binary can not be found in your PATH!",
-                                value)
+                    LOG.debug("'%s' binary can not be found in your PATH!",
+                              value)
                     continue
 
                 self.__analyzers[name] = os.path.realpath(compiler_binary)
diff --git a/analyzer/codechecker_analyzer/analyzers/analyzer_types.py b/analyzer/codechecker_analyzer/analyzers/analyzer_types.py
index b400f55a..4832788c 100644
--- a/analyzer/codechecker_analyzer/analyzers/analyzer_types.py
+++ b/analyzer/codechecker_analyzer/analyzers/analyzer_types.py
@@ -13,6 +13,7 @@ Supported analyzer types.
 
 import os
 import re
+import sys
 
 from codechecker_analyzer import env
 from codechecker_common.logger import get_logger
@@ -114,6 +115,27 @@ def is_z3_refutation_capable(context):
                                                  analyzer_env)
 
 
+def print_unsupported_analyzers(errored):
+    """ Print error messages which occured during analyzer detection. """
+    for analyzer_binary, reason in errored:
+        LOG.warning("Analyzer '%s' is enabled but CodeChecker is failed to "
+                    "execute analysis with it: '%s'. Please check your "
+                    "'PATH' environment variable and the "
+                    "'config/package_layout.json' file!",
+                    analyzer_binary, reason)
+
+
+def check_available_analyzers(analyzers, errored):
+    """ Handle use case when no analyzer can be found on the user machine. """
+    if analyzers:
+        return
+
+    print_unsupported_analyzers(errored)
+    LOG.error("Failed to run command because no analyzers can be found on "
+              "your machine!")
+    sys.exit(1)
+
+
 def check_supported_analyzers(analyzers, context):
     """
     Checks the given analyzers in the current context for their executability
@@ -137,7 +159,7 @@ def check_supported_analyzers(analyzers, context):
     for analyzer_name in analyzers:
         if analyzer_name not in supported_analyzers:
             failed_analyzers.add((analyzer_name,
-                                  "Analyzer unsupported by CodeChecker."))
+                                  "Analyzer unsupported by CodeChecker!"))
             continue
 
         # Get the compiler binary to check if it can run.
@@ -145,8 +167,8 @@ def check_supported_analyzers(analyzers, context):
         analyzer_bin = analyzer_binaries.get(analyzer_name)
         if not analyzer_bin:
             failed_analyzers.add((analyzer_name,
-                                  "Failed to detect analyzer binary."))
-            available_analyzer = False
+                                  "Failed to detect analyzer binary!"))
+            continue
         elif not os.path.isabs(analyzer_bin):
             # If the analyzer is not in an absolute path, try to find it...
             found_bin = supported_analyzers[analyzer_name].\\
@@ -171,7 +193,7 @@ def check_supported_analyzers(analyzers, context):
             # Analyzers unavailable under absolute paths are deliberately a
             # configuration problem.
             failed_analyzers.add((analyzer_name,
-                                  "Cannot execute analyzer binary."))
+                                  "Cannot execute analyzer binary!"))
             available_analyzer = False
 
         if available_analyzer:
diff --git a/analyzer/codechecker_analyzer/cmd/analyze.py b/analyzer/codechecker_analyzer/cmd/analyze.py
index b2d9af51..e9944d89 100644
--- a/analyzer/codechecker_analyzer/cmd/analyze.py
+++ b/analyzer/codechecker_analyzer/cmd/analyze.py
@@ -943,8 +943,11 @@ def main(args):
     analyzer_clang_binary = \\
         context.analyzer_binaries.get(
             clangsa.analyzer.ClangSA.ANALYZER_NAME)
-    analyzer_clang_version = clangsa.version.get(analyzer_clang_binary,
-                                                 analyzer_env)
+
+    analyzer_clang_version = None
+    if analyzer_clang_binary:
+        analyzer_clang_version = clangsa.version.get(analyzer_clang_binary,
+                                                     analyzer_env)
 
     actions, skipped_cmp_cmd_count = log_parser.parse_unique_log(
         compile_commands,
diff --git a/analyzer/codechecker_analyzer/cmd/checkers.py b/analyzer/codechecker_analyzer/cmd/checkers.py
index 1cabde70..ae205880 100644
--- a/analyzer/codechecker_analyzer/cmd/checkers.py
+++ b/analyzer/codechecker_analyzer/cmd/checkers.py
@@ -217,6 +217,7 @@ def main(args):
     working_analyzers, errored = analyzer_types.check_supported_analyzers(
         args.analyzers,
         context)
+    analyzer_types.check_available_analyzers(working_analyzers, errored)
 
     analyzer_environment = env.extend(context.path_env_extra,
                                       context.ld_lib_path_extra)
@@ -400,8 +401,4 @@ def main(args):
     if rows:
         print(twodim.to_str(args.output_format, header, rows))
 
-    for analyzer_binary, reason in errored:
-        LOG.error("Failed to get checkers for '%s'!"
-                  "The error reason was: '%s'", analyzer_binary, reason)
-        LOG.error("Please check your installation and the "
-                  "'config/package_layout.json' file!")
+    analyzer_types.print_unsupported_analyzers(errored)