Yes, it is a violation. Legally, you can go elsewhere--to a shelter, for example, if you have no friend, family member, etc. who will take you in, no one will take you as a roommate for an amount you can afford, etc. Because you are not legally required to live with the drug users and do have options, even if only bad ones (e.g. a shelter), the law considers you to be living with the drug users of your own free will; and living with known drug users of your own free will is a probation violation.