If it's not exactly the way they think it should be too bad for them. This is only a temporary situation. When you've achieved your dreams as a writer and when you're receiving an income from your writings, you'll have all the time in the world to be whatever everybody else thinks you should be. For now they'll just have to be patient. Feeling guilty will prevent you from writing.
Successful writers keep on writing, no matter what.
Put the guilt, the housework and everything that may distract you from writing, on hold. You're doing the best you can and that is all that matters. Your kids will not die, starve, be morally corrupted, or suffer any form of long lasting personality disorder because you have chosen to seek out a career that will be of benefit to them in the end. Just remember to tell them that you love them as you sit and type away. Let's be realistic here and not exaggerate.