No, not really.  She probably can not deny you visitation with the children based upon that reason. I say "probably" because it is a judgement call for BOTH of you.  You would not want to put your kids in jeopardy if they were not well either.  But kids get runny noses this time of year at the drop of a hat.  And your place is a s good as hers to nurse them back to health.  Good luck.