It will lead to moisture contamination in your brake reservoir where the brake fluid is stored. When you don't change your brake fluid, your cars braking capabilities suffer greatly for it leading to poor braking performance.