Repository: angr/angr

Issue:
Return address not properly initialized by `project.factory.call_state()` when passing the `stack_base` argument
*Disclaimer: The angr suite is maintained by a small team of volunteers. While we cannot guarantee any timeliness for fixes and enhancements, we will do our best. For more real-time help with angr, from us and the community, join our [Slack.](http://angr.io/invite/)*

---

**Describe the bug**
Please include a clear and concise description of what the bug is.

Ordinarily, `project.factory.call_state()` initializes the return address of the called state to be the address of a SimProcedure that ends up ensuring that any states which reach the end of the function are put into the `deadended` stash.

When explicitly passing the `stack_base` parameter to `project.factory.call_state()`, this does not happen properly, and instead the stack location where the return value should be is simply uninitialized memory.

This issue occurs whether or not the `alloc_base` parameter is explicitly passed, or left as default.

**Environment Information**
Many common issues are caused by problems with the local Python environment.
Before submitting, double-check that your versions of all modules in the angr suite (angr, cle, pyvex, ...) are up to date.
Please include the output of `python -m angr.misc.bug_report` here.

```
angr environment report
=============================
Date: 2019-03-25 15:32:19.144606
Running in virtual environment at /Users/craig/sunjay/pypy
Platform: macosx-10.14-x86_64
Python version: 3.6.1 (dab365a465140aa79a5f3ba4db784c4af4d5c195, Feb 18 2019, 10:53:27)
[PyPy 7.0.0-alpha0 with GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)]
######## angr #########
Python found it in /Users/craig/sunjay/pypy/site-packages/angr
Pip version angr 8.19.2.4
######## ailment #########
Python found it in /Users/craig/sunjay/pypy/site-packages/ailment
Pip version ailment 8.19.2.4
######## cle #########
Python found it in /Users/craig/sunjay/pypy/site-packages/cle
Pip version cle 8.19.2.4
######## pyvex #########
Python found it in /Users/craig/sunjay/pypy/site-packages/pyvex
Pip version pyvex 8.19.2.4
######## claripy #########
Python found it in /Users/craig/sunjay/pypy/site-packages/claripy
Pip version claripy 8.19.2.4
######## archinfo #########
Python found it in /Users/craig/sunjay/pypy/site-packages/archinfo
Pip version archinfo 8.19.2.4
######## ana #########
Python found it in /Users/craig/sunjay/pypy/site-packages/ana
Pip version ana 0.5
######## z3 #########
Python found it in /Users/craig/sunjay/pypy/site-packages/z3
Pip version not found!
######## unicorn #########
Python found it in /Users/craig/sunjay/pypy/site-packages/unicorn
Pip version unicorn 1.0.1
######### Native Module Info ##########
angr: None
unicorn: <CDLL '/Users/craig/sunjay/pypy/site-packages/unicorn/lib/libunicorn.dylib', handle 7fe0575f6840 at 0x1130650c0>
pyvex: <cffi.api._make_ffi_library.<locals>.FFILibrary object at 0x000000011268f2b8>
z3: <CDLL '/Users/craig/sunjay/pypy/site-packages/z3/lib/libz3.dylib', handle 7fe05c950dc0 at 0x116fde640>
```

**To Reproduce**
Please include *both a script to reproduce the crash, and attach the binary used, if possible*

The `testbinary` used in the below session is attached here as [testbinary.gz](https://github.com/angr/angr/files/3005767/testbinary.gz). (Apologies, I had to gzip the binary as GitHub doesn't seem to allow raw executable binaries to be attached to issues.)

[Edit: The issue is probably unrelated to the specific binary used; see [first comment below](https://github.com/angr/angr/issues/1485#issuecomment-476410317)]

```
>>>> proj = angr.Project('testbinary')
WARNING | 2019-03-25 15:35:11,700 | cle.loader | The main binary is a position-independent executable. It is being loaded with a base address of 0x400000.
>>>> funcaddr = proj.loader.find_symbol('crypto_sign_ed25519_tweet_keypair').rebased_addr
>>>> state = proj.factory.call_state(funcaddr, claripy.BVS("arg1", 64), claripy.BVS("arg2", 64))
>>>> state2 = proj.factory.call_state(funcaddr, claripy.BVS("arg1", 64), claripy.BVS("arg2", 64), stack_base=0x7ffffffffffefff0)
>>>> state.regs.rsp
<BV64 0x7fffffffffefff8>
>>>> state2.regs.rsp
<BV64 0x7ffffffffff0000>
>>>> state.mem[state.regs.rsp].uint64_t
<uint64_t <BV64 0x1000078> at <BV64 0x7fffffffffefff8>>
>>>> state2.mem[state2.regs.rsp].uint64_t
WARNING | 2019-03-25 15:19:55,577 | angr.state_plugins.symbolic_memory | The program is accessing memory or registers with an unspecified value. This could indicate unwanted behavior.
WARNING | 2019-03-25 15:19:55,578 | angr.state_plugins.symbolic_memory | angr will cope with this by generating an unconstrained symbolic variable and continuing. You can resolve this by:
WARNING | 2019-03-25 15:19:55,578 | angr.state_plugins.symbolic_memory | 1) setting a value to the initial state
WARNING | 2019-03-25 15:19:55,578 | angr.state_plugins.symbolic_memory | 2) adding the state option ZERO_FILL_UNCONSTRAINED_{MEMORY,REGISTERS}, to make unknown regions hold null
WARNING | 2019-03-25 15:19:55,578 | angr.state_plugins.symbolic_memory | 3) adding the state option SYMBOL_FILL_UNCONSTRAINED_{MEMORY_REGISTERS}, to suppress these messages.
WARNING | 2019-03-25 15:19:55,578 | angr.state_plugins.symbolic_memory | Filling memory at 0x7ffffffffff0000 with 8 unconstrained bytes
<uint64_t <BV64 mem_7ffffffffff0000_4_64{UNINITIALIZED}> at <BV64 0x7ffffffffff0000>>
```

**Additional context**
Add any other context about the problem here.

Beginning of `readelf -e` on the attached binary:
```
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x10b0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          62424 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         11
  Size of section headers:           64 (bytes)
  Number of section headers:         29
  Section header string table index: 28
```


Discussion:
Note: this issue appears to be independent of the actual binary used.  E.g., I can also reproduce the behavior with the `echo` binary from `angr-binaries/tests/x86_64` and the function `program_invocation_name`:

```
>>>> proj = angr.Project('../angr-binaries/tests/x86_64/echo', auto_load_libs=False)
>>>> funcaddr = proj.loader.find_symbol('program_invocation_name').rebased_addr
...
```
@twizmwazin, another part of code for you to dive into :-)
Nah, this one is mine. I'm halfway in already.

On Mon, Mar 25, 2019 at 6:03 PM Yan <notifications@github.com> wrote:

> @twizmwazin <https://github.com/twizmwazin>, another part of code for you
> to dive into :-)
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/angr/angr/issues/1485#issuecomment-476434005>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/ACYg9aJSl2dHqSdMCKa7iJnJqxwwQLmmks5vaXH1gaJpZM4cJtxF>
> .
>


Resolving patch:
diff --git a/angr/calling_conventions.py b/angr/calling_conventions.py
--- a/angr/calling_conventions.py
+++ b/angr/calling_conventions.py
@@ -30,21 +30,29 @@ def __init__(self, value):
 
 
 class AllocHelper:
-    def __init__(self, ptr, grow_like_stack, reverse_result):
-        self.ptr = ptr
-        self.grow_like_stack = grow_like_stack
+    def __init__(self, ptrsize, reverse_result):
+        self.base = claripy.BVS('alloc_base', ptrsize)
+        self.ptr = self.base
         self.reverse_result = reverse_result
+        self.stores = {}
 
     def dump(self, val, state, endness='Iend_BE'):
-        if self.grow_like_stack:
-            self.ptr -= val.length // state.arch.byte_width
-            state.memory.store(self.ptr, val, endness=endness)
-            return self.ptr.reversed if self.reverse_result else self.ptr
-        else:
-            state.memory.store(self.ptr, val, endness=endness)
-            out = self.ptr
-            self.ptr += val.length // state.arch.byte_width
-            return out.reversed if self.reverse_result else out
+        self.stores[self.ptr.cache_key] = (val, endness)
+        out = self.ptr
+        self.ptr += val.length // state.arch.byte_width
+        return out.reversed if self.reverse_result else out
+
+    def translate(self, val, base):
+        return val.replace(self.base, base)
+
+    def apply(self, state, base):
+        for ptr, (val, endness) in self.stores.items():
+            state.memory.store(self.translate(ptr.ast, base), self.translate(val, base), endness=endness)
+
+    def size(self):
+        val = self.translate(self.ptr, claripy.BVV(0, len(self.ptr)))
+        assert val.op == 'BVV'
+        return abs(val.args[0])
 
 
 class SimFunctionArgument:
@@ -526,26 +534,31 @@ def setup_callsite(self, state, ret_addr, args, stack_base=None, alloc_base=None
         binary format to be placed into simulated memory. Lists (representing arrays) must be entirely elements of the
         same type and size, while tuples (representing structs) can be elements of any type and size.
         If you'd like there to be a pointer to a given value, wrap the value in a `PointerWrapper`. Any value
-        that can't fit in a register will be automatically put in a
-        PointerWrapper.
+        that can't fit in a register will be automatically put in a PointerWrapper.
 
         If stack_base is not provided, the current stack pointer will be used, and it will be updated.
-        If alloc_base is not provided, the current stack pointer will be used, and it will be updated.
-        You might not like the results if you provide stack_base but not alloc_base.
+        If alloc_base is not provided, the stack base will be used and grow_like_stack will implicitly be True.
 
         grow_like_stack controls the behavior of allocating data at alloc_base. When data from args needs to be wrapped
         in a pointer, the pointer needs to point somewhere, so that data is dumped into memory at alloc_base. If you
-        set alloc_base to point to somewhere other than the stack, set grow_like_stack to False so that sequencial
+        set alloc_base to point to somewhere other than the stack, set grow_like_stack to False so that sequential
         allocations happen at increasing addresses.
         """
 
+        # STEP 0: clerical work
+
         if isinstance(self, SimCCSoot):
             SimEngineSoot.setup_callsite(state, args, ret_addr)
             return
 
-        allocator = AllocHelper(alloc_base if alloc_base is not None else state.regs.sp,
-                grow_like_stack,
-                self.arch.memory_endness == 'Iend_LE')
+        allocator = AllocHelper(self.arch.bits, self.arch.memory_endness == 'Iend_LE')
+
+        #
+        # STEP 1: convert all values into serialized form
+        # this entails creating the vals list of simple values to store and also populating the allocator's
+        # understanding of what aux data needs to be stored
+        # This is also where we compute arg locations (arg_locs)
+        #
 
         if self.func_ty is not None:
             vals = [self._standardize_value(arg, ty, state, allocator.dump) for arg, ty in zip(args, self.func_ty.args)]
@@ -568,16 +581,47 @@ def setup_callsite(self, state, ret_addr, args, stack_base=None, alloc_base=None
                     vals[i] = claripy.BVV(0, state.arch.bits - val.length).concat(val)
             arg_locs[i] = arg_session.next_arg(is_fp=False, size=vals[i].length // state.arch.byte_width)
 
-        if alloc_base is None:
-            state.regs.sp = allocator.ptr
+        #
+        # STEP 2: decide on memory storage locations
+        # implement the contract for stack_base/alloc_base/grow_like_stack
+        # after this, stack_base should be the final stack pointer, alloc_base should be the final aux storage location,
+        # and the stack pointer should be updated
+        #
 
         if stack_base is None:
+            if alloc_base is None:
+                alloc_size = allocator.size()
+                state.regs.sp -= alloc_size
+                alloc_base = state.regs.sp
+                grow_like_stack = False
+
             state.regs.sp -= self.stack_space(arg_locs)
 
             # handle alignment
             alignment = (state.regs.sp + self.STACKARG_SP_DIFF) % self.STACK_ALIGNMENT
             state.regs.sp -= alignment
 
+        else:
+            state.regs.sp = stack_base
+
+            if alloc_base is None:
+                alloc_base = stack_base + self.stack_space(arg_locs)
+                grow_like_stack = False
+
+        if grow_like_stack:
+            alloc_base -= allocator.size()
+        if type(alloc_base) is int:
+            alloc_base = claripy.BVV(alloc_base, state.arch.bits)
+
+        for i, val in enumerate(vals):
+            vals[i] = allocator.translate(val, alloc_base)
+
+        #
+        # STEP 3: store everything!
+        #
+
+        allocator.apply(state, alloc_base)
+
         for loc, val in zip(arg_locs, vals):
             if val.length > loc.size * 8:
                 raise ValueError("Can't fit value {} into location {}".format(repr(val), repr(loc)))
diff --git a/angr/simos/simos.py b/angr/simos/simos.py
--- a/angr/simos/simos.py
+++ b/angr/simos/simos.py
@@ -206,6 +206,8 @@ def state_call(self, addr, *args, **kwargs):
         grow_like_stack = kwargs.pop('grow_like_stack', True)
 
         if state is None:
+            if stack_base is not None:
+                kwargs['stack_end'] = (stack_base + 0x1000) & ~0xfff
             state = self.state_blank(addr=addr, **kwargs)
         else:
             state = state.copy()


Test patch:
diff --git a/tests/test_callable.py b/tests/test_callable.py
--- a/tests/test_callable.py
+++ b/tests/test_callable.py
@@ -1,6 +1,7 @@
 import nose
 import angr
 import claripy
+import archinfo
 from angr.sim_type import SimTypePointer, SimTypeFunction, SimTypeChar, SimTypeInt, parse_defns
 from angr.errors import AngrCallableMultistateError
 
@@ -146,6 +147,20 @@ def test_callable_c_manyfloatsum():
     for arch in addresses_manysum:
         yield run_callable_c_manysum, arch
 
+def test_setup_callsite():
+    p = angr.load_shellcode(b'b', arch=archinfo.ArchX86())
+
+    s = p.factory.call_state(0, "hello", stack_base=0x1234, alloc_base=0x5678, grow_like_stack=False)
+    assert (s.regs.sp == 0x1234).is_true()
+    assert (s.mem[0x1234 + 4].long.resolved == 0x5678).is_true()
+    assert (s.memory.load(0x5678, 5) == b'hello').is_true()
+
+    s = p.factory.call_state(0, "hello", stack_base=0x1234)
+    assert (s.regs.sp == 0x1234).is_true()
+    assert (s.mem[0x1234 + 4].long.resolved == 0x1234 + 8).is_true()
+    assert (s.memory.load(0x1234 + 8, 5) == b'hello').is_true()
+
+
 
 if __name__ == "__main__":
     print('testing manyfloatsum with symbolic arguments')