Repository: angr/angr

Issue:
Incorrect stack alignment on x86-64
I'm trying to get my rust binary running under `angr` for symbolic execution.  I've pared it down to, well, nothing:
```
#![feature(alloc_system, global_allocator, allocator_api)]

extern crate alloc_system;

use alloc_system::System;

#[global_allocator]
static A: System = System;

fn main() {
    return;
}
```
(We use the system allocator here instead of `jemalloc`, Rust's default, to avoid having to patch out a bunch of syscalls.)

You can build this binary by starting a new crate with `cargo init --bin` and using that as the `src/main.rs` file.  LMK if you'd like more detail for building the rust side, of course.  (Or I can send you a binary.)

Then, I'm running this script inside the `angr/angr` docker container:
```
import angr

def main(p):
    proj = angr.Project(
        p,
        use_sim_procedures=True,
        translation_cache=True,
        support_selfmodifying_code=False,
    )

    # No-op the signal handler installation (for now)
    class signal(angr.SimProcedure):
        def run(self, *args, **kwargs):
            return 0

    proj.hook_symbol('signal', signal())

    opts = {
        angr.options.ZERO_FILL_UNCONSTRAINED_MEMORY,
        angr.options.STRICT_PAGE_ACCESS,
        angr.options.UNICORN,
    }
    entry_state = proj.factory.entry_state(
        add_options=opts,
        args={},
        env={},
    )

    simulation_manager = angr.SimulationManager(
        project=proj,
        active_states=[entry_state],
    )

    i = 0
    while len(simulation_manager.active) == 1 and not simulation_manager.errored:
        simulation_manager.step()
        i += 1

    if simulation_manager.active:
        print("Stopped with %d states!" % len(simulation_manager.active))
        actions = list(simulation_manager.active[0].history.actions)
        print actions[-1]

    elif simulation_manager.errored:
        errored = simulation_manager.errored[0]
        print("Errored: %s" % (errored.error,))
        state = errored.state
        print("rip: %s" % (state.regs.rip))
        print("rsp: %s" % (state.regs.rsp))
        print("rbp: %s" % (state.regs.rbp))
        proj.factory.block(state.solver.eval(state.regs.rip)).pp()
        print("")

        for frame in state.callstack:
            try:
                name = proj.kb.labels.get(frame.current_function_address)
            except KeyError:
                name = None
            print(">> %s" % (name,))
        print("")
        import ipdb; ipdb.set_trace()
```
The program steps through until...
```
Errored: Cannot execute following jumpkind Ijk_SigSEGV
rip: <BV64 0x407f54>
rsp: <BV64 0x7fffffffffefdb8>
rbp: <BV64 0x7fffffffffefe98>
0x407f54:	movaps	xmmword ptr [rbp - 0xc0], xmm0
0x407f5b:	movaps	xmmword ptr [rbp - 0xd0], xmm0
0x407f62:	movaps	xmmword ptr [rbp - 0xe0], xmm0
0x407f69:	mov	qword ptr [rbp - 0xb0], 0
0x407f74:	lea	rdi, [rbp - 0xe0]
0x407f7b:	call	0x4032c0

>> _ZN3std3sys4unix6thread5guard4init17ha4e308f7b00e4ed6E
>> _ZN3std2rt19lang_start_internal17hd640de3aab7bd269E
>> main
>> __libc_start_main
>> None
```
Note that `rbp` is not 16 byte aligned, so the `movaps` is going to fail with a protection fault.

If I run the same program directly, it works, and if I break at the offending instruction in `gdb`, `rbp` is indeed 16 byte aligned.

## Configuration
```
$ uname -a
Linux sujayakar-vm 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ rustc --version
rustc 1.27.0-nightly (79252ff4e 2018-04-29)
```
Incorrect stack alignment on x86-64
I'm trying to get my rust binary running under `angr` for symbolic execution.  I've pared it down to, well, nothing:
```
#![feature(alloc_system, global_allocator, allocator_api)]

extern crate alloc_system;

use alloc_system::System;

#[global_allocator]
static A: System = System;

fn main() {
    return;
}
```
(We use the system allocator here instead of `jemalloc`, Rust's default, to avoid having to patch out a bunch of syscalls.)

You can build this binary by starting a new crate with `cargo init --bin` and using that as the `src/main.rs` file.  LMK if you'd like more detail for building the rust side, of course.  (Or I can send you a binary.)

Then, I'm running this script inside the `angr/angr` docker container:
```
import angr

def main(p):
    proj = angr.Project(
        p,
        use_sim_procedures=True,
        translation_cache=True,
        support_selfmodifying_code=False,
    )

    # No-op the signal handler installation (for now)
    class signal(angr.SimProcedure):
        def run(self, *args, **kwargs):
            return 0

    proj.hook_symbol('signal', signal())

    opts = {
        angr.options.ZERO_FILL_UNCONSTRAINED_MEMORY,
        angr.options.STRICT_PAGE_ACCESS,
        angr.options.UNICORN,
    }
    entry_state = proj.factory.entry_state(
        add_options=opts,
        args={},
        env={},
    )

    simulation_manager = angr.SimulationManager(
        project=proj,
        active_states=[entry_state],
    )

    i = 0
    while len(simulation_manager.active) == 1 and not simulation_manager.errored:
        simulation_manager.step()
        i += 1

    if simulation_manager.active:
        print("Stopped with %d states!" % len(simulation_manager.active))
        actions = list(simulation_manager.active[0].history.actions)
        print actions[-1]

    elif simulation_manager.errored:
        errored = simulation_manager.errored[0]
        print("Errored: %s" % (errored.error,))
        state = errored.state
        print("rip: %s" % (state.regs.rip))
        print("rsp: %s" % (state.regs.rsp))
        print("rbp: %s" % (state.regs.rbp))
        proj.factory.block(state.solver.eval(state.regs.rip)).pp()
        print("")

        for frame in state.callstack:
            try:
                name = proj.kb.labels.get(frame.current_function_address)
            except KeyError:
                name = None
            print(">> %s" % (name,))
        print("")
        import ipdb; ipdb.set_trace()
```
The program steps through until...
```
Errored: Cannot execute following jumpkind Ijk_SigSEGV
rip: <BV64 0x407f54>
rsp: <BV64 0x7fffffffffefdb8>
rbp: <BV64 0x7fffffffffefe98>
0x407f54:	movaps	xmmword ptr [rbp - 0xc0], xmm0
0x407f5b:	movaps	xmmword ptr [rbp - 0xd0], xmm0
0x407f62:	movaps	xmmword ptr [rbp - 0xe0], xmm0
0x407f69:	mov	qword ptr [rbp - 0xb0], 0
0x407f74:	lea	rdi, [rbp - 0xe0]
0x407f7b:	call	0x4032c0

>> _ZN3std3sys4unix6thread5guard4init17ha4e308f7b00e4ed6E
>> _ZN3std2rt19lang_start_internal17hd640de3aab7bd269E
>> main
>> __libc_start_main
>> None
```
Note that `rbp` is not 16 byte aligned, so the `movaps` is going to fail with a protection fault.

If I run the same program directly, it works, and if I break at the offending instruction in `gdb`, `rbp` is indeed 16 byte aligned.

## Configuration
```
$ uname -a
Linux sujayakar-vm 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ rustc --version
rustc 1.27.0-nightly (79252ff4e 2018-04-29)
```


Discussion:
You can change the initial stack pointer (in this case, lowering it by 8) by modifying archinfo.arch_x64.py. Do you want to give it a try and let us know if that solves the problem?
Interestingly not!  Right after constructing the project, I added
```
proj.arch.initial_sp -= 8
```
but it still hits the same unaligned memory read.  There must be something the compiler emitted to align it that's behaving differently here -- I'll continue to dig.
I'm pretty sure that's actually because `Arch.initial_sp` is copied from there into another field of the arch info (default_register_values or something) at _class creation time_, not construction time or even use time. Besides that, it would be important to fix this in angr instead of archinfo because we may add a variable amount of data to the stack during state construction and it's important to align after that in all cases. There's some comparable code that does stack alignment for `call_state` but I'd need to dig it up and find it. I'll do this when I'm not feeling sick.
Hi, digging this up since I've stumbled upon the same problem.
I believe the issue is in the [implementation of `__libc_start_main`](https://github.com/angr/angr/blob/master/angr/procedures/glibc/__libc_start_main.py#L136), where it says:

```python
        if isinstance(self.state.arch, ArchAMD64):
            # (rsp+8) must be aligned to 16 as required by System V ABI
            # ref: http://www.x86-64.org/documentation/abi.pdf , page 16
            self.state.regs.rsp = (self.state.regs.rsp & 0xfffffffffffffff0) - 8
```

Reading the document (which now lives at https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x86-64-psABI-1.0.pdf), it says

> In other words, the value (%rsp + 8) is always a multiple of 16 (32 or 64) when control is transferred to the function entry point

I believe the key phrase here is "_when control is transferred to the function entry point_", which I read as "**after** the `call` instruction". However, in angr's `__libc_start_main` abstraction, the alignment is made **before** the return pointer is placed onto the stack. Thus, when the control is transferred to the function entry point, alignment is off by 8.

Does this seem plausible? If so I'm happy to provide a pull request.
Interesting. I would call this plausible indeed. If it fixes something for you, we'd be happy to take it as a PR.
You can change the initial stack pointer (in this case, lowering it by 8) by modifying archinfo.arch_x64.py. Do you want to give it a try and let us know if that solves the problem?
Interestingly not!  Right after constructing the project, I added
```
proj.arch.initial_sp -= 8
```
but it still hits the same unaligned memory read.  There must be something the compiler emitted to align it that's behaving differently here -- I'll continue to dig.
I'm pretty sure that's actually because `Arch.initial_sp` is copied from there into another field of the arch info (default_register_values or something) at _class creation time_, not construction time or even use time. Besides that, it would be important to fix this in angr instead of archinfo because we may add a variable amount of data to the stack during state construction and it's important to align after that in all cases. There's some comparable code that does stack alignment for `call_state` but I'd need to dig it up and find it. I'll do this when I'm not feeling sick.
Hi, digging this up since I've stumbled upon the same problem.
I believe the issue is in the [implementation of `__libc_start_main`](https://github.com/angr/angr/blob/master/angr/procedures/glibc/__libc_start_main.py#L136), where it says:

```python
        if isinstance(self.state.arch, ArchAMD64):
            # (rsp+8) must be aligned to 16 as required by System V ABI
            # ref: http://www.x86-64.org/documentation/abi.pdf , page 16
            self.state.regs.rsp = (self.state.regs.rsp & 0xfffffffffffffff0) - 8
```

Reading the document (which now lives at https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x86-64-psABI-1.0.pdf), it says

> In other words, the value (%rsp + 8) is always a multiple of 16 (32 or 64) when control is transferred to the function entry point

I believe the key phrase here is "_when control is transferred to the function entry point_", which I read as "**after** the `call` instruction". However, in angr's `__libc_start_main` abstraction, the alignment is made **before** the return pointer is placed onto the stack. Thus, when the control is transferred to the function entry point, alignment is off by 8.

Does this seem plausible? If so I'm happy to provide a pull request.
Interesting. I would call this plausible indeed. If it fixes something for you, we'd be happy to take it as a PR.

Resolving patch:
diff --git a/angr/procedures/glibc/__libc_start_main.py b/angr/procedures/glibc/__libc_start_main.py
--- a/angr/procedures/glibc/__libc_start_main.py
+++ b/angr/procedures/glibc/__libc_start_main.py
@@ -134,9 +134,11 @@ def run(self, main, argc, argv, init, fini):
 
     def after_init(self, main, argc, argv, init, fini, exit_addr=0):
         if isinstance(self.state.arch, ArchAMD64):
-            # (rsp+8) must be aligned to 16 as required by System V ABI
-            # ref: http://www.x86-64.org/documentation/abi.pdf , page 16
-            self.state.regs.rsp = (self.state.regs.rsp & 0xfffffffffffffff0) - 8
+            # (rsp+8) must be aligned to 16 as required by System V ABI.
+            # Since call will place a return address (8 bytes) onto the stack,
+            # align rsp to 16 bytes here.
+            # ref: https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x86-64-psABI-1.0.pdf , page 18
+            self.state.regs.rsp = (self.state.regs.rsp & 0xfffffffffffffff0)
         self.call(self.main, (self.argc, self.argv, self.envp), 'after_main')
 
     def after_main(self, main, argc, argv, init, fini, exit_addr=0):


Test patch: