When you do the math, you may find that the dollar difference between working while paying the nanny and staying at home is minimal. Don't forget about the incidental costs of a job. There's commuting, lunches out, and those work clothes you might otherwise not buy. If you make the decision to stay at home, but worry about your career or your sanity, hang on. You can do something about both.
Some jobs lend themselves to part-time work.
Telecommuting and job-sharing are increasingly common in the workplace. If you decide not to work for pay and crave that all-important adult interaction, consider volunteering. It involves more than chaperoning field trips. Serious organizational skills are required for fundraising, for example. If you do go back to work, you'll have a variety of activities that are resume-worthy.