Take top-down into consideration on holding based movement