The fear of doing the right thing haunts me. Consider the ages of your children and their needs, as well as your own needs and desires. You are working part-time. Talk with your husband about this decision. You are a team.
Life is about change.
You can devote your time and energy to your children, particularly in the first three years of development, and rest assured that if you are content to do so, they will benefit from your availability. You will get to know who they are from an early age. Your children will eventually and gradually grow out of their dependency on you. As they grow older, they begin friendships and relationships of their own. Though you remain primary to them, you do recede in importance.