Remove unnecessary mux in the critical path