Repository: angr/angr

Issue:
Decoding error occurred when handling `ud2` instruction sometimes
<!--
*Disclaimer:
The angr suite is maintained by a small team of volunteers.
While we cannot guarantee any timeliness for fixes and enhancements, we will do our best.
For more real-time help with angr, from us and the community, join our [Slack.](http://angr.io/invite/)*
-->
---

**Describe the bug.**
<!--
Please include a clear and concise description of what the bug is.
-->
angr raise `Decoding error` when meeting `ud2` instruction sometimes.

**Environment Information.**
<!--
Many common issues are caused by problems with the local Python environment.
Before submitting, double-check that your versions of all modules in the angr suite (angr, cle, pyvex, ...) are up to date.
Please include the output of `python -m angr.misc.bug_report` here.
-->
- Ubuntu18.04-amd64.
```
Platform: linux-x86_64
Python version: 3.7.5 (default, Nov  7 2019, 10:50:52)
[GCC 8.3.0]
######## angr #########
Platform: linux-x86_64
Python version: 3.7.5 (default, Nov  7 2019, 10:50:52)
[GCC 8.3.0]
######## angr #########
Python found it in /home/binpang/binary_reasemble/relatedWork/angr-dev/angr/angr
Pip version angr 8.20.1.7
Git info:
        Current commit 2a0e77229b3cf2f85938faa26dcf3a0e451559af from branch master
        Checked out from remote origin: https://github.com/angr/angr
######## ailment #########
Python found it in /home/binpang/binary_reasemble/relatedWork/angr-dev/ailment/ailment
Pip version ailment 8.20.1.7
Git info:
        Current commit c696edc9f31ecccaaa36b1e40ca1bed6c9715242 from branch master
        Checked out from remote origin: https://github.com/angr/ailment
######## cle #########
Python found it in /home/binpang/binary_reasemble/relatedWork/angr-dev/cle/cle
Pip version cle 8.20.1.7
Git info:
        Current commit 58ceebbc2fd9697fada9f5beeb07047b158549ca from branch master
        Checked out from remote origin: https://github.com/angr/cle
######## pyvex #########
Python found it in /home/binpang/binary_reasemble/relatedWork/angr-dev/pyvex/pyvex
Pip version pyvex 8.20.1.7
Git info:
        Current commit f4753f85a8c18dae9dbb32301f8a7804efd36491 from branch master
        Checked out from remote origin: https://github.com/angr/pyvex
######## claripy #########
Python found it in /home/binpang/binary_reasemble/relatedWork/angr-dev/claripy/claripy
Pip version claripy 8.20.1.7
Git info:
        Current commit b6eabc37aed7c2989faf12406a247004b5849216 from branch master
        Checked out from remote origin: https://github.com/angr/claripy
######## archinfo #########
Python found it in /home/binpang/binary_reasemble/relatedWork/angr-dev/archinfo/archinfo
Pip version archinfo 8.20.1.7
Git info:
        Current commit 1db8e3c200e9346b2902b9d929bbd2676113b72e from branch master
        Checked out from remote origin: https://github.com/angr/archinfo
######## z3 #########
Python found it in /home/binpang/.virtualenvs/angr/lib/python3.7/site-packages/z3
Pip version z3-solver 4.8.7.0
Couldn't find git info
######## unicorn #########
Python found it in /home/binpang/.virtualenvs/angr/lib/python3.7/site-packages/unicorn
Pip version unicorn 1.0.1
Couldn't find git info
######### Native Module Info ##########
angr: <CDLL '/home/binpang/binary_reasemble/relatedWork/angr-dev/angr/angr/lib/angr_native.so', handle 3053f70 at 0x7f806b342b50>
unicorn: <CDLL '/home/binpang/.virtualenvs/angr/lib/python3.7/site-packages/unicorn/lib/libunicorn.so', handle 27feb60 at 0x7f806a29de90>
pyvex: <cffi.api._make_ffi_library.<locals>.FFILibrary object at 0x7f806b2ad290>
z3: <CDLL '/home/binpang/.virtualenvs/angr/lib/python3.7/site-packages/z3/lib/libz3.so', handle 2a28840 at 0x7f8068077f50>
```

**To Reproduce.**
<!--
Please include *both a script to reproduce the crash, and attach the binary used, if possible*
-->

Script:
```python
"""
file: angrBlocks.py
"""
import angr
import logging
import optparse

logging.getLogger('angr.analyses').setLevel(logging.DEBUG)
def dumpBlocks(binary):
    # "force_complete_scan" default is True
    p = angr.Project(binary, load_options={'auto_load_libs': False})
    cfg = p.analyses.CFGFast()

    # iter over the cfg functions
    for func_addr in cfg.functions:
        func = cfg.functions[func_addr]
        print("function %s, its addr is 0x%x" % (func.name, func.addr))
        # iter over blocks
        for bb in func.blocks:
            print("basic block addr 0x%x, its size 0x%x" % (bb.addr, bb.size))
            cfg_node = cfg.get_any_node(bb.addr)
            # bb.instruction_addrs can get the instrction address of block
            if cfg_node != None and bb.size != 0:
                for inst in bb.capstone.insns:
                    inst_va = inst.address
                    print("instruction: 0x%x" % (inst_va))

if __name__ == "__main__":
    parser = optparse.OptionParser()
    parser.add_option("-b", "--binary", dest = "binary", action = "store", type = "string", \
            help = "binary file", default = None)
    (options, args) = parser.parse_args()
    if options.binary == None:
        print("please input the binary file")
        exit(-1)

    dumpBlocks(options.binary)
```

The binary file that trigger the bug is attached. Before disassembling, I strip the binary file firstly.  
[binary with symbol](https://github.com/angr/angr/files/4205202/dwp.zip)

**Additional context.**
<!--
Add any other context about the problem here.
-->

angr's debug log shows that it raise `Decoding error` at `ud2` instruction address, such like:

```
ERROR   | 2020-02-14 10:39:40,059 | ^[[34mangr.analyses.cfg.cfg_fast^[[0m | ^[[34mDecoding error occurred at address 0x809b1f7 of function 0x809b1f0.^[[0m
ERROR   | 2020-02-14 10:39:40,248 | ^[[34mangr.analyses.cfg.cfg_fast^[[0m | ^[[34mDecoding error occurred at address 0x80952c7 of function 0x80952c0.^[[0m
```

Disassembled instruction at `0x809b1f7` is `ud2`:
```
0809b1f0 <_ZN4gold16Elf_reloc_mapperD0Ev>:
 809b1f0:       55                      push   %ebp
 809b1f1:       89 e5                   mov    %esp,%ebp
 809b1f3:       50                      push   %eax
 809b1f4:       8b 45 08                mov    0x8(%ebp),%eax
 809b1f7:       0f 0b                   ud2
 809b1f9:       89 45 fc                mov    %eax,-0x4(%ebp)
 809b1fc:       8d 74 26 00             lea    0x0(%esi,%eiz,1),%esi
```
Same problem at address `0x80952c7`.


Discussion:
That is expected. `ud2` literally means `undefined instruction`. VEX does not lift it to anything but raise a decoding error. CFGFast will then mark it as an invalid instruction and will treat those two bytes as data. This behavior seems OK to me.
Hi, thanks for your replying.

But it seems that angr doesn't mark `0x809b1f7-0x809b1f8` as invalid instruction here. As angr "force-scans" from `0x809b1f8`.
```
DEBUG   | 2020-02-24 11:38:28,644 | ^[[34mangr.analyses.cfg.cfg_fast^[[0m | ^[[34mReturning a new recon address: 0x809b1f8^[[0m
DEBUG   | 2020-02-24 11:38:28,644 | ^[[34mangr.analyses.cfg.cfg_fast^[[0m | ^[[34mSearching address 809b1f8^[[0m
DEBUG   | 2020-02-24 11:38:28,644 | ^[[34mangr.analyses.cfg.cfg_fast^[[0m | ^[[34mForce-scanning to 0x809b1f8^[[0m
```

The log file is attached here:
[angr_debug](https://github.com/angr/angr/files/4245679/angr_debug.log.zip)


Resolving patch:
diff --git a/angr/analyses/cfg/cfg_fast.py b/angr/analyses/cfg/cfg_fast.py
--- a/angr/analyses/cfg/cfg_fast.py
+++ b/angr/analyses/cfg/cfg_fast.py
@@ -3403,6 +3403,7 @@ def _generate_cfgnode(self, cfg_job, current_function_addr):
             nodecode = False
             irsb = None
             irsb_string = None
+            lifted_block = None
             try:
                 lifted_block = self._lift(addr, size=distance, opt_level=self._iropt_level, collect_data_refs=True,
                                           strict_block_end=True)
@@ -3462,21 +3463,22 @@ def _generate_cfgnode(self, cfg_job, current_function_addr):
                 else:
                     irsb_size = irsb.size
                 # special handling for ud, ud1, and ud2 on x86 and x86-64
-                if is_x86_x64_arch \
-                        and len(irsb_string) >= 2 \
-                        and irsb_string[-2:] in {
+                if irsb_string[-2:] == b'\x0f\x0b' and self.project.arch.name == 'AMD64':
+                    # VEX supports ud2 and make it part of the block size, only in AMD64.
+                    valid_ins = True
+                    nodecode_size = 0
+                elif lifted_block is not None \
+                        and is_x86_x64_arch \
+                        and len(lifted_block.bytes) - irsb_size > 2 \
+                        and lifted_block.bytes[irsb_size : irsb_size + 2] in {
                             b'\x0f\xff',  # ud0
                             b'\x0f\xb9',  # ud1
                             b'\x0f\x0b',  # ud2
                         }:
                     # ud0, ud1, and ud2 are actually valid instructions.
                     valid_ins = True
-                    if irsb_string[-2:] == b'\x0f\x0b':
-                        # VEX supports ud2 and make it part of the block size.
-                        nodecode_size = 0
-                    else:
-                        # VEX does not support ud0 or ud1. they are not part of the block size.
-                        nodecode_size = 2
+                    # VEX does not support ud0 or ud1 or ud2 under AMD64. they are not part of the block size.
+                    nodecode_size = 2
                 else:
                     valid_ins = False
                     nodecode_size = 1


Test patch:
diff --git a/tests/test_vex.py b/tests/test_vex.py
--- a/tests/test_vex.py
+++ b/tests/test_vex.py
@@ -334,12 +334,25 @@ def test_amd64_ud012_behaviors():
     block_1 = a.factory.block(0)
     assert block_1.size == 2
 
-    # according to VEX, ud2 *is* part of the block
+    # according to VEX, ud2 under AMD64 *is* part of the block
     a = load_shellcode(b"\x90\x90\x0f\x0b", "amd64")
     block_2 = a.factory.block(0)
     assert block_2.size == 4
 
 
+def test_x86_ud2_behaviors():
+
+    # Test if VEX's lifter behaves as what CFGFast expects
+    #
+    # Note: if such behaviors change in the future, you also need to fix the ud2 handling logic in
+    # CFGFast._generate_cfgnode().
+
+    # according to VEX, ud2 on x86 is not part of the block
+    a = load_shellcode(b"\x90\x90\x0f\x0b", "x86")
+    block_0 = a.factory.block(0)
+    assert block_0.size == 2
+
+
 if __name__ == '__main__':
     g = globals().copy()
     for func_name, func in g.items():