While this depends on the relationship you have with your husband and the dynamic between you two, in general, having friends of the opposite sex is perfectly acceptable and should not be cause for concern on its own.