Yes, you can control by imposing conditions on the parenting plan or if the court imposed any such conditions on the joint custodial order. But if that is not the case, you cannot control who will be around your child when they are with the other parent.