Generally, replacing all windows at once is more cost effective. Plus, it adds to a uniform aesthetic. However, this choice depends on your budget.