It is true that this might anger him further. Abusers can be very unpredictable. But, allowing him to stay involved in your life will almost certainly create a situation where he can continue to abuse you and affect your daughter. I would go even further and suggest that, after you get custody, you try to leave the state and get as far away from him as possible, if you can do that legally. If he says he has changed, be very skeptical.
Abusers can change, but it requires a lot of time and hard work.
Most abusers can't change significantly without professional help. An apology means nothing. Most abusers feel genuine remorse after an episode, because they know they are at risk of losing their victim and perhaps being held responsible for their actions. On one level they know what they are doing is wrong, but on a deeper level, it doesn't matter. Abusers can use words, actions and emotions to get what they want.