The children may really not want to spend time with the other parent, sometimes for good reason. But, this is actually quite rare. The children may want to spend time with the other parent but not want to leave the parent they're with. It is common for human beings to simultaneously desire two, mutually exclusive results. We often wish we could eat our cake and have it too.
Children are no different.
In fact, most children really want to be with both parents and not to leave either. Their favorite fantasy is that their parents would get back together. The children may be sensing non-verbal cues from the parent they are leaving that she or he is sad when the children leave. The children maybe reflecting the parent's feelings, not expressing their own. The children may believe it pleases the parent for them to be sad about leaving.