We should account for the view direction changing