Your kids will mis you, but if they are with someone they have gotten to know before you leave, someone who is responsible and likes kids, they will do fine. Are you involved in alocal church or social club. We often swap kids with friends. We watch their kids while they get away and they watch ours when we go. If you see this as a problem do something about it.
Every married couple needs time together without kids and distractions to keep the passion alive.
A perfectly good compromise for this situation that I haven't seen suggested is that you take the kids along. You'll have to come up with the fare and perhaps accomodations, but that might be an option. I like Mudder's suggestion of a nanny as well. In the risk analysis business, I'd tell you to concern yourself with things that have a high probability of happening, and causing serious harm. And divorce is on top of the list.