You have him. Don't push him away. Don't make your entire relationship about the Ex and the Step-Children. While it's sometimes hard to do, you and your husband are a couple in your own right, without the children. She is not your ex wife, she is his problem.
Men seem to be better at compartmentalizing their lives than women.
It is a skill I recommend we all try to learn. Leave work at work and the Ex at her house. When the children are not with you focus on yourselves as a couple. You're lucky, in tact parents don't get to do this. Enjoy it while you can because when you have your own children life will change more than you realize.