Repository: ericsson/codechecker
Issue: 2nd analysis fails when source file has been removed

Hi,

I tried to analyse https://github.com/flok99/Baislicka-2.0

First time I did on 2d7dc07dbfa0f86b2dbce897867bb4e3bc679ca7 (git hash):
CodeChecker check -b 'make -f mak' -o ~/results/ && CodeChecker store ~/results -n chess

Then I ran the following on 8a5fa3e50b2a96628fcda71b0c8d18a92d8730db:
CodeChecker check -b 'make -f mak' -o ~/results/ && CodeChecker store ~/results -n chess

which resulted in:

(CodeChecker venv) [sa@clang src]$ CodeChecker check -b 'make -f mak' -o ~/results/ 2>&1 | tee ~/0002-chess.log
[INFO 2021-02-04 15:01] - Starting build ...
cc    -c -o attack.o attack.c
cc    -c -o bitboards.o bitboards.c
cc    -c -o display.o display.c
cc    -c -o eval.o eval.c
cc    -c -o fen.o fen.c
cc    -c -o hashtable.o hashtable.c
cc    -c -o main.o main.c
cc    -c -o move.o move.c
cc    -c -o movegen.o movegen.c
cc    -c -o perft.o perft.c
cc    -c -o search.o search.c
cc    -c -o uci.o uci.c
cc -o bla attack.o bitboards.o display.o eval.o fen.o hashtable.o main.o move.o movegen.o perft.o search.o uci.o -pthread
[INFO 2021-02-04 15:01] - Build finished successfully.
[INFO 2021-02-04 15:01] - Enabled checkers:
clangsa: apiModeling.google.GTest, core.CallAndMessage, core.DivideZero, core.DynamicTypePropagation, core.NonNullParamChecker, core.NullDereference, core.StackAddressEscape, core.UndefinedBinaryOperatorResult, core.VLASize, core.builtin.BuiltinFunctions, core.builtin.NoReturnFunctions, core.uninitialized.ArraySubscript, core.uninitialized.Assign, core.uninitialized.Branch, core.uninitialized.CapturedBlockVariable, core.uninitialized.UndefReturn, cplusplus.NewDelete, cplusplus.NewDeleteLeaks, cplusplus.SelfAssignment, deadcode.DeadStores, nullability.NullPassedToNonnull, nullability.NullReturnedFromNonnull, optin.cplusplus.VirtualCall, optin.portability.UnixAPI, security.FloatLoopCounter, security.insecureAPI.UncheckedReturn, security.insecureAPI.getpw, security.insecureAPI.gets, security.insecureAPI.mkstemp, security.insecureAPI.mktemp, security.insecureAPI.rand, security.insecureAPI.vfork, unix.API, unix.Malloc, unix.MallocSizeof, unix.MismatchedDeallocator, unix.StdCLibraryFunctions, unix.Vfork, unix.cstring.BadSizeArg, unix.cstring.NullArg, valist.CopyToSelf, valist.Uninitialized, valist.Unterminated
clang-tidy: bugprone-suspicious-memset-usage, bugprone-undefined-memory-manipulation, cert-dcl54-cpp, cert-dcl58-cpp, cert-dcl59-cpp, cert-err09-cpp, cert-fio38-c, cert-oop11-cpp, cppcoreguidelines-special-member-functions, google-build-namespaces, google-global-names-in-headers, misc-assert-side-effect, misc-bool-pointer-implicit-conversion, misc-dangling-handle, misc-definitions-in-headers, misc-fold-init-type, misc-forward-declaration-namespace, misc-forwarding-reference-overload, misc-inaccurate-erase, misc-incorrect-roundings, misc-inefficient-algorithm, misc-lambda-function-name, misc-macro-parentheses, misc-macro-repeated-side-effects, misc-misplaced-const, misc-misplaced-widening-cast, misc-move-const-arg, misc-move-constructor-init, misc-move-forwarding-reference, misc-redundant-expression, misc-sizeof-container, misc-sizeof-expression, misc-string-compare, misc-string-constructor, misc-string-literal-with-embedded-nul, misc-suspicious-enum-usage, misc-suspicious-missing-comma, misc-suspicious-semicolon, misc-swapped-arguments, misc-unconventional-assign-operator, misc-uniqueptr-reset-release, misc-unused-raii, misc-use-after-move, misc-virtual-near-miss
[INFO 2021-02-04 15:01] - Starting static analysis ...
[INFO 2021-02-04 15:01] - [1/24] clangsa analyzed attack.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/attack.c_clangsa_9b607539d1090470e3d42a148037a66c.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [2/24] clangsa analyzed bitboards.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/bitboards.c_clangsa_87c09ce8c44e1c72c2ec8135e48dbd22.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [3/24] clangsa analyzed display.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/display.c_clangsa_b9d1d2354f703c89e7beec8502113f47.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [4/24] clangsa analyzed eval.c successfully.
[INFO 2021-02-04 15:01] - [5/24] clangsa analyzed fen.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/fen.c_clangsa_6949e986ebb34d80a02769cc3cc59221.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [6/24] clangsa analyzed hashtable.c successfully.
[INFO 2021-02-04 15:01] - [7/24] clangsa analyzed main.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/main.c_clangsa_875054fa7bb0836796ae2e927403aa30.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [8/24] clangsa analyzed move.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/move.c_clangsa_ef4e4f0e075682f1207833610c70a9bc.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [9/24] clangsa analyzed movegen.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/movegen.c_clangsa_a49084a64ca888e25ccf5bad87531594.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [10/24] clangsa analyzed perft.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/perft.c_clangsa_161bff93bcda01ab5b068caf90459a99.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [11/24] clangsa analyzed search.c successfully.
[INFO 2021-02-04 15:01] - [12/24] clangsa analyzed uci.c successfully.
[INFO 2021-02-04 15:01] - [13/24] clang-tidy analyzed attack.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/attack.c_clang-tidy_9b607539d1090470e3d42a148037a66c.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [14/24] clang-tidy analyzed bitboards.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/bitboards.c_clang-tidy_87c09ce8c44e1c72c2ec8135e48dbd22.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [15/24] clang-tidy analyzed display.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/display.c_clang-tidy_b9d1d2354f703c89e7beec8502113f47.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [16/24] clang-tidy analyzed eval.c successfully.
[INFO 2021-02-04 15:01] - [17/24] clang-tidy analyzed fen.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/fen.c_clang-tidy_6949e986ebb34d80a02769cc3cc59221.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [18/24] clang-tidy analyzed hashtable.c successfully.
[INFO 2021-02-04 15:01] - [19/24] clang-tidy analyzed main.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/main.c_clang-tidy_875054fa7bb0836796ae2e927403aa30.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [20/24] clang-tidy analyzed move.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/move.c_clang-tidy_ef4e4f0e075682f1207833610c70a9bc.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [21/24] clang-tidy analyzed movegen.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/movegen.c_clang-tidy_a49084a64ca888e25ccf5bad87531594.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [22/24] clang-tidy analyzed perft.c successfully.
[WARNING 2021-02-04 15:01] - Previous analysis results in '/usr/home/sa/results/perft.c_clang-tidy_161bff93bcda01ab5b068caf90459a99.plist' has been overwritten.
[INFO 2021-02-04 15:01] - [23/24] clang-tidy analyzed search.c successfully.
[INFO 2021-02-04 15:01] - [24/24] clang-tidy analyzed uci.c successfully.
[INFO 2021-02-04 15:01] - ----==== Summary ====----
[INFO 2021-02-04 15:01] - Successfully analyzed
[INFO 2021-02-04 15:01] -   clangsa: 12
[INFO 2021-02-04 15:01] -   clang-tidy: 12
[INFO 2021-02-04 15:01] - Reanalyzed compilation commands: 16
[INFO 2021-02-04 15:01] - Total analyzed compilation commands: 12
[INFO 2021-02-04 15:01] - ----=================----
[INFO 2021-02-04 15:01] - Analysis finished.
[INFO 2021-02-04 15:01] - To view results in the terminal use the "CodeChecker parse" command.
[INFO 2021-02-04 15:01] - To store results use the "CodeChecker store" command.
[INFO 2021-02-04 15:01] - See --help and the user guide for further options about parsing and storing the reports.
[INFO 2021-02-04 15:01] - ----=================----
[INFO 2021-02-04 15:01] - Analysis length: 10.954143285751343 sec.
[INFO 2021-02-04 15:01] - Remove plist file '/usr/home/sa/results/movegen_white.c_clang-tidy_16d9c45fb4fdaab21506f9d84a554766.plist' because it refers to a source file ('/usr/home/sa/baislicka/src/movegen_white.c') which was removed.
[ERROR 2021-02-04 15:01] - Running check failed.
Traceback (most recent call last):
  File "/usr/home/sa/codechecker/build/CodeChecker/cc_bin/../lib/python3/codechecker_analyzer/cmd/check.py", line 822, in main
    analysis_exit_status = analyze_module.main(analyze_args)
  File "/usr/home/sa/codechecker/build/CodeChecker/lib/python3/codechecker_analyzer/cmd/analyze.py", line 1024, in main
    __cleanup_metadata(metadata_prev, metadata)
  File "/usr/home/sa/codechecker/build/CodeChecker/lib/python3/codechecker_analyzer/cmd/analyze.py", line 762, in __cleanup_metadata
    del metadata['result_source_files'][plist_file]
KeyError: 'result_source_files'
Traceback (most recent call last):
  File "/usr/home/sa/codechecker/build/CodeChecker/cc_bin/../lib/python3/codechecker_analyzer/cmd/check.py", line 822, in main
    analysis_exit_status = analyze_module.main(analyze_args)
  File "/usr/home/sa/codechecker/build/CodeChecker/lib/python3/codechecker_analyzer/cmd/analyze.py", line 1024, in main
    __cleanup_metadata(metadata_prev, metadata)
  File "/usr/home/sa/codechecker/build/CodeChecker/lib/python3/codechecker_analyzer/cmd/analyze.py", line 762, in __cleanup_metadata
    del metadata['result_source_files'][plist_file]
KeyError: 'result_source_files'
Traceback (most recent call last):
  File "/usr/home/sa/codechecker/build/CodeChecker/cc_bin/CodeChecker.py", line 141, in main
    sys.exit(args.func(args))
  File "/usr/home/sa/codechecker/build/CodeChecker/cc_bin/../lib/python3/codechecker_analyzer/cmd/check.py", line 853, in main
    return analysis_exit_status
UnboundLocalError: local variable 'analysis_exit_status' referenced before assignment


Changed files:
analyzer/codechecker_analyzer/cmd/analyze.py

diff --git a/analyzer/codechecker_analyzer/cmd/analyze.py b/analyzer/codechecker_analyzer/cmd/analyze.py
index b2d9af51..ffe479cf 100644
--- a/analyzer/codechecker_analyzer/cmd/analyze.py
+++ b/analyzer/codechecker_analyzer/cmd/analyze.py
@@ -761,12 +761,23 @@ def __cleanup_metadata(metadata_prev, metadata):
                 LOG.info("Remove plist file '%s' because it refers to a "
                          "source file ('%s') which was removed.",
                          plist_file, source_file)
-                del metadata['result_source_files'][plist_file]
+                __del_result_source_file(metadata, plist_file)
                 os.remove(plist_file)
             except OSError:
                 LOG.warning("Failed to remove plist file: %s", plist_file)
 
 
+def __del_result_source_file(metadata, file_path):
+    """ Remove file path from metadata result source files. """
+    if 'result_source_files' in metadata:
+        # Kept for backward-compatibility reason.
+        del metadata['result_source_files'][file_path]
+    else:
+        for tool in metadata.get('tools', {}):
+            result_source_files = tool.get('result_source_files', {})
+            del result_source_files[file_path]
+
+
 def __get_result_source_files(metadata):
     """ Get result source files from the given metadata. """
     if 'result_source_files' in metadata: