Repository: angr/angr

Issue:
CFG Party Foul: Function prologue false-positive causes function fragmentation
*Disclaimer: The angr suite is maintained by a small team of volunteers. While we cannot guarantee any timeliness for fixes and enhancements, we will do our best. For more real-time help with angr, from us and the community, join our [Slack.](http://angr.io/invite/)*

---

**Describe the bug**
Please include a clear and concise description of what the bug is.

In `mbed_copy_nvic` (ox08000b99) of `tests/armel/Nucleo_read_hyperterminal-stripped.elf`
we mis-identify the function prologue (WTF is GCC even doing???) and lose an instruction at the beginning of the function.  The function is called directly, by `software_init_hook`, which is called eventually from the entry point.

My suspicion about what's going on here is yet another ordering thing regarding function recovery.  software_init_hook is discovered via indirect jump resolution, as its address is loaded out of what would be a relocation.  This is one of those things our fixed-up IJ resolver handles.  However, by the time IJs get resolved, we have already found the wrong function start point, and things get messy.

Obviously this is specific to the forthcoming features of fix/arm_cfg_party_2, as we have thumb prologues turned on.

**Environment Information**
Many common issues are caused by problems with the local Python environment.
Before submitting, double-check that your versions of all modules in the angr suite (angr, cle, pyvex, ...) are up to date.
Please include the output of `python -m angr.misc.bug_report` here.
`fix/arm_cfg_party_2`

**To Reproduce**
Please include *both a script to reproduce the crash, and attach the binary used, if possible*
```
p = angr.Project("Nucleo_read_hyperterminal-stripped.elf")                                                       
cfg = p.analyses.CFGFast(normalize=True, collect_data_references=True, force_complete_scan=False, resolve_indirect_jumps=True, detect_tail_calls=True) 
f = p.kb.functions[0x08000b99]
for bl in f.blocks():
    bl.pp()
```

Should be more than just the ldr instruction
Check out the non-stripped version of the binary in the repo to see what i mean

**Additional context**
Add any other context about the problem here.



Discussion:
I found more examples of this:
`_read`, `_write`, `_close` and `_lseek` have the same issue, and are found in different circumstances.

```
In [12]: f = p.kb.functions['_read']

In [13]: for bl in sorted(f.blocks, key=lambda x: x.addr):
    ...:     bl.pp()
    ...:     
0x800083d:      cmp     r0, #2
```

Looks like maybe a pipelining optimization?

Resolving patch:
diff --git a/angr/analyses/cfg/cfg_base.py b/angr/analyses/cfg/cfg_base.py
--- a/angr/analyses/cfg/cfg_base.py
+++ b/angr/analyses/cfg/cfg_base.py
@@ -514,6 +514,49 @@ def remove_edge(self, block_from, block_to):
         if edge in self._graph:
             self._graph.remove_edge(*edge)
 
+    def _merge_cfgnodes(self, cfgnode_0, cfgnode_1):
+        """
+        Merge two adjacent CFGNodes into one.
+
+        :param CFGNode cfgnode_0:   The first CFGNode.
+        :param CFGNode cfgnode_1:   The second CFGNode.
+        :return:                    None
+        """
+
+        assert cfgnode_0.addr + cfgnode_0.size == cfgnode_1.addr
+        addr0, addr1 = cfgnode_0.addr, cfgnode_1.addr
+        new_node = cfgnode_0.copy()
+        new_node.size += cfgnode_1.size
+
+        # Update the graph and the nodes dict accordingly
+        if addr1 in self._nodes_by_addr:
+            self._nodes_by_addr[addr1].remove(cfgnode_1)
+            if not self._nodes_by_addr[addr1]:
+                del self._nodes_by_addr[addr1]
+        del self._nodes[cfgnode_1.block_id]
+
+        self._nodes_by_addr[addr0].remove(cfgnode_0)
+        if not self._nodes_by_addr[addr0]:
+            del self._nodes_by_addr[addr0]
+        del self._nodes[cfgnode_0.block_id]
+
+        in_edges = list(self._graph.in_edges(cfgnode_0, data=True))
+        out_edges = list(self._graph.out_edges(cfgnode_1, data=True))
+
+        self._graph.remove_node(cfgnode_0)
+        self._graph.remove_node(cfgnode_1)
+
+        self._graph.add_node(new_node)
+        for src, _, data in in_edges:
+            self._graph.add_edge(src, new_node, **data)
+        for _, dst, data in out_edges:
+            self._graph.add_edge(new_node, dst, **data)
+
+        # Put the new node into node dicts
+        self._nodes[new_node.block_id] = new_node
+        self._nodes_by_addr[addr0].append(new_node)
+
+
     def _to_snippet(self, cfg_node=None, addr=None, size=None, thumb=False, jumpkind=None, base_state=None):
         """
         Convert a CFGNode instance to a CodeNode object.
@@ -1389,12 +1432,14 @@ def make_functions(self):
                                                                  called_function_addrs,
                                                                  blockaddr_to_function
                                                                  )
-        removed_functions_b = self._process_irrational_function_starts(tmp_functions,
-                                                                       called_function_addrs,
-                                                                       blockaddr_to_function
-                                                                       )
+        removed_functions_b, adjusted_cfgnodes = self._process_irrational_function_starts(tmp_functions,
+                                                                                          called_function_addrs,
+                                                                                          blockaddr_to_function
+                                                                                          )
         removed_functions = removed_functions_a | removed_functions_b
 
+        # Remove all nodes that are adjusted
+        function_nodes.difference_update(adjusted_cfgnodes)
         for n in self.graph.nodes():
             if n.addr in tmp_functions or n.addr in removed_functions:
                 function_nodes.add(n)
@@ -1639,6 +1684,7 @@ def _process_irrational_function_starts(self, functions, predetermined_function_
         If the following conditions are met, we will remove the second function and merge it into the first function:
         - The second function is not called by other code.
         - The first function has only one jumpout site, which points to the second function.
+        - The first function and the second function are adjacent.
 
         :param FunctionManager functions:   All functions that angr recovers.
         :return:                            A set of addresses of all removed functions.
@@ -1647,6 +1693,7 @@ def _process_irrational_function_starts(self, functions, predetermined_function_
 
         addrs = sorted(functions.keys())
         functions_to_remove = set()
+        adjusted_cfgnodes = set()
 
         for addr_0, addr_1 in zip(addrs[:-1], addrs[1:]):
 
@@ -1678,11 +1725,25 @@ def _process_irrational_function_starts(self, functions, predetermined_function_
                 if target_addr != addr_1:
                     continue
 
+                # Are func_0 adjacent to func_1?
+                if block.addr + block.size != addr_1:
+                    continue
+
                 l.debug("Merging function %#x into %#x.", addr_1, addr_0)
 
+                # Merge block addr_0 and block addr_1
+                cfgnode_0 = self.get_any_node(addr_0)
+                cfgnode_1 = self.get_any_node(addr_1)
+                self._merge_cfgnodes(cfgnode_0, cfgnode_1)
+                adjusted_cfgnodes.add(cfgnode_0)
+                adjusted_cfgnodes.add(cfgnode_1)
+
                 # Merge it
                 func_1 = functions[addr_1]
                 for block_addr in func_1.block_addrs:
+                    if block_addr == addr_1:
+                        # Skip addr_1 (since it has been merged to the preceding block)
+                        continue
                     merge_with = self._addr_to_function(addr_0, blockaddr_to_function, functions)
                     blockaddr_to_function[block_addr] = merge_with
 
@@ -1691,7 +1752,7 @@ def _process_irrational_function_starts(self, functions, predetermined_function_
         for to_remove in functions_to_remove:
             del functions[to_remove]
 
-        return functions_to_remove
+        return functions_to_remove, adjusted_cfgnodes
 
     def _addr_to_function(self, addr, blockaddr_to_function, known_functions):
         """
diff --git a/angr/analyses/cfg/cfg_fast.py b/angr/analyses/cfg/cfg_fast.py
--- a/angr/analyses/cfg/cfg_fast.py
+++ b/angr/analyses/cfg/cfg_fast.py
@@ -2685,6 +2685,7 @@ def _indirect_jump_unresolved(self, jump):
         dst_node = CFGNode(self._unresolvable_target_addr, 0, self,
                            function_address=self._unresolvable_target_addr,
                            simprocedure_name='UnresolvableTarget',
+                           block_id=self._unresolvable_target_addr,
                            )
 
         # add the dst_node to self._nodes
@@ -2771,6 +2772,7 @@ def _remove_redundant_overlapping_blocks(self):
                                                                     ),
                                             thumb=a.thumb,
                                             byte_string=None if a.byte_string is None else a.byte_string[nop_length:],
+                                            block_id=next_node_addr,
                                             )
                         self.graph.add_node(next_node)
 
@@ -2931,7 +2933,8 @@ def _shrink_node(self, node, new_size, remove_function=True):
                                                      if node.addr <= i < node.addr + new_size
                                                    ),
                            thumb=node.thumb,
-                           byte_string=None if node.byte_string is None else node.byte_string[:new_size]
+                           byte_string=None if node.byte_string is None else node.byte_string[:new_size],
+                           block_id=node.addr,
                            )
 
         old_in_edges = self.graph.in_edges(node, data=True)
diff --git a/angr/analyses/cfg/cfg_node.py b/angr/analyses/cfg/cfg_node.py
--- a/angr/analyses/cfg/cfg_node.py
+++ b/angr/analyses/cfg/cfg_node.py
@@ -1,10 +1,13 @@
 import traceback
+import logging
 
 import archinfo
 
 from ...codenode import BlockNode, HookNode
 from ...engines.successors import SimSuccessors
 
+_l = logging.getLogger(__name__)
+
 
 class CFGNodeCreationFailure(object):
     """
@@ -79,6 +82,11 @@ def __init__(self,
         self.has_return = False
         self._hash = None
 
+        # Sanity check
+        if self.block_id is None and type(self) is CFGNode:  # pylint: disable=unidiomatic-typecheck
+            _l.warning("block_id is unspecified for %s. Default to its address %#x.", str(self), self.addr)
+            self.block_id = self.addr
+
     @property
     def name(self):
         if self._name is None:


Test patch: