Typically the walker holder does not need to be removed. Since it is made of a flexible fabric, you should be able to fold the back down without having to remove the walker holder.