Handle deref and reference properly in IR-based cross compiler