Simplify displace handling

Should be faster now