Refine a lot of logics in finding columns for AggResultColumns