Yes. The word is babymoon and there are many couples who have witnessed this as one of the best experiences ever. There are many couples who prefer to travel with their babies and go on a honeymoon due to either a time crunch or financial restraints.