Still, the point of being your own boss is to gain control over your life, not to become the personal slave of every client. With input from your wife, you can lay out together just what constitutes a client emergency that merits sacrificing precious private time. Most people arrive at a set of rules after being burned or because they've seen someone else handling the matter in a way they admire. Draw up rules that you're willing to obey. And tell all your current clients how you've had to reorient.
Irritation is understandable, but vent it in the right direction.
Take a jog or a fast walk, write in a journal, or phone a friend and unload. Spend a few minutes playing with the baby. Having made your rules, make peace with the sacrifices your company demands. Weigh them against how much you enjoy what you're doing, how important it is to you that you're actively pursuing your goal or vision. That's the trade-off.