This would not be "per se" defamation. Defamation is the making of a false statement of fact to other people, which statement damages your reputation or causes you some loss or injury. Truth is a defense to defamation. IF you were on drugs at the time, then clearly it is not defamation to say that you were on drugs, since in this case, the statement is true. Also, opinions, even if harmful, are not defamation. If you appeared (e.g. based on your pupils or other physical signs) to be on drugs and they only said that you "appeared" (or something to that effect) to be on drugs, that is also not defamation, since their statement that to them, you "appeared" to be on drugs is only an opinion (their subjective impression), and opinions are not defamation. Only if they said something definitively factual, such as "he/she IS on illegal drugs" when in fact you were not, might this be defamation.