Repository: angr/angr

Issue:
SimEngineError raised because control flows to locations determined by the input PoV
**Describe the bug.**

When tracing a CGC binary using it's PoV, control flows to invalid memory locations without any code at the location causing the tracing to stop with a `SimEngineError`. A cursory analysis of concrete execution of the CGC binary using the PoV in qemu reveals the crash is expected behaviour: the input somehow influences the control flow and causes the binary to crash.

**Environment Information.**

Platform: linux-x86_64
Python version: 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0]
######## angr #########
Python found it in /home/dnivra/angr-dev/angr/angr
Pip version angr 9.0.gitrolling
Git info:
        Current commit 762becbf9c66d4798b8c23cfa512a7f893e2bcf9 from branch master
        Checked out from remote origin: https://github.com/angr/angr
######## ailment #########
Python found it in /home/dnivra/angr-dev/ailment/ailment
Pip version ailment 9.0.gitrolling
Git info:
        Current commit 4e2bba6f0299d1eda6ae570ceabd91eb8a0c72be from branch master
        Checked out from remote origin: https://github.com/angr/ailment
######## cle #########
Python found it in /home/dnivra/angr-dev/cle/cle
Pip version cle 9.0.gitrolling
Git info:
        Current commit 80dcd50abfaa70cbd5b2e360fe41b71406acbfb4 from branch master
        Checked out from remote origin: https://github.com/angr/cle
######## pyvex #########
Python found it in /home/dnivra/angr-dev/pyvex/pyvex
Pip version pyvex 9.0.gitrolling
Git info:
        Current commit 969ec1f10d3e3b15407ee986052aa4b6f2e9df05 from branch master
        Checked out from remote origin: https://github.com/angr/pyvex
######## claripy #########
Python found it in /home/dnivra/angr-dev/claripy/claripy
Pip version claripy 9.0.gitrolling
Git info:
        Current commit 34f31c487f7453f4666cd6fd1d529f417ff6ca08 from branch master
        Checked out from remote origin: https://github.com/angr/claripy
######## archinfo #########
Python found it in /home/dnivra/angr-dev/archinfo/archinfo
Pip version archinfo 9.0.gitrolling
Git info:
        Current commit 437b194538ccb0bf118b4b674613b88832b0b342 from branch master
        Checked out from remote origin: https://github.com/angr/archinfo
######## z3 #########
Python found it in /home/dnivra/.virtualenvs/angr-dev/lib/python3.8/site-packages/z3
Pip version z3-solver 4.8.9.0
Couldn't find git info
######## unicorn #########
Python found it in /home/dnivra/.virtualenvs/angr-dev/lib/python3.8/site-packages/unicorn
Pip version unicorn 1.0.2rc4
Couldn't find git info
######### Native Module Info ##########
angr: <CDLL '/home/dnivra/angr-dev/angr/angr/lib/angr_native.so', handle 2cd33e0 at 0x7fe5c9a93220>
unicorn: <CDLL '/home/dnivra/.virtualenvs/angr-dev/lib/python3.8/site-packages/unicorn/lib/libunicorn.so', handle 2407f70 at 0x7fe5cf5cf1c0>
pyvex: <cffi.api._make_ffi_library.<locals>.FFILibrary object at 0x7fe5d01de130>
z3: <CDLL '/home/dnivra/.virtualenvs/angr-dev/lib/python3.8/site-packages/z3/lib/libz3.so', handle 26679d0 at 0x7fe5cccdf340>

**To Reproduce.**

[angr-bug-2664-error-repro-1.zip](https://github.com/angr/angr/files/6481693/angr-bug-2664-error-repro-1.zip) 
[angr-bug-2664-error-repro-2.zip](https://github.com/angr/angr/files/6481694/angr-bug-2664-error-repro-2.zip) are two cases to reproduce this issue. In former, control ends up at 0x0(the value is read from a symbolic address). In latter, control ends up at 0xb7ffdb04(tracing this takes quite a while. Edit: ~30 mins till `SimEngineError` occurs) and then crashes. I'm not sure how control ends up there: I observe the same when running the binary on concrete PoV bytes in qemu.


Discussion:
I've pushed https://github.com/angr/angr/pull/2665, which fixes the issue for  me. I don't know if it's a correct fix, but it makes sense to me.
My "fix" causes CI to fail, so I'm going to have to kick this over to @Kyle-Kyle. What should we do to make it so that crash windup doesn't throw an exception for this binary? I would think it should be the linked patch, but that seems to disagree with our tests. Do we need to adjust rex for this?
In our current tracing model, I think this is the expected behavior: we currently only handle cases where control flow is *directly* affected by input.
In these two cases, the control flow is *indirectly* affected by input.
In case 1, the control flow is controlled by where the symbolic address points to, which is something we can't handle atm. (case2 is too complicated, but I think it has something to do with symbolic addresses too)

Unwinding 1 more step does fix this specific case but breaks other cases because it unwinds to a state whose successors do not crash for our current testcases. More concerning: this problem has the context-sensitivity issue: the symbolic address can be an address to a nested struct and the program dereferences the struct to find a pointer and finally jump to it. This means: we may need to unwind several steps to finally get back to the "pre-crashing state" where the program starts dereferencing symbolic addresses(?).

In fact, how do we define "pre-crashing state" in this case? I suggest we use the state where the program starts dereferencing symbolic addresses because those addresses can be invalid in the first place. And also this is needed if we eventually want to exploit it.

So, that means:
1. we define pre-crashing state as where the program starts dereferencing symbolic addresses
2. 1 implies pre-crashing state may take many steps to actual control flow hijacking

To fix it, we may add a concretization strategy and enable it in tracer. And ofc, fix rex because rex assumes crash is 1 step away starting from the "pre-crashing state"

@rhelmot ideas?
I'm not particularly interested in the "right" way to do this; we don't need to exploit this binary, only to trace it. The thing I'm confused about is why our windup needs to execute the crashing instruction. I thought the point was that the windup step sets the instruction pointer to the crash address, and the final step actually executes that instruction, triggering a crash. In this case, the crash is "jump to 0" and the crash address is the address of the jump to 0. What other cases are there where you need to execute further than that?
iirc, we execute the pre-crashing state with pre-constraints removed so we can get an unconstrained state
yes... I know that, but I feel like my patch is an implementation of doing that.
now I understand what you were saying. Yep, your fix looks correct but somehow breaks CI. I will have a look at it tmr.

Resolving patch:
diff --git a/angr/exploration_techniques/tracer.py b/angr/exploration_techniques/tracer.py
--- a/angr/exploration_techniques/tracer.py
+++ b/angr/exploration_techniques/tracer.py
@@ -900,7 +900,7 @@ def crash_windup(cls, state, crash_addr):
         if inst_cnt == 0:
             insts = 0
         elif crash_addr in inst_addrs:
-            insts = inst_addrs.index(crash_addr) + 1
+            insts = inst_addrs.index(crash_addr)
         else:
             insts = inst_cnt - 1
 
@@ -921,12 +921,13 @@ def crash_windup(cls, state, crash_addr):
         l.debug("reconstraining... ")
         state.preconstrainer.reconstrain()
 
+        l.debug("final step...")
+        succs = state.step(num_inst=1)
+
         # now remove our breakpoints since other people might not want them
         state.inspect.remove_breakpoint("address_concretization", bp1)
         state.inspect.remove_breakpoint("address_concretization", bp2)
 
-        l.debug("final step...")
-        succs = state.step()
         successors = succs.flat_successors + succs.unconstrained_successors
         crash_state = successors[0]
         return last_state, crash_state


Test patch: