Premise: Bugger or "buggar" is a slang word. In the United Kingdom, the term is a general-purpose expletive, used to imply dissatisfaction, or to refer to someone or something whose behaviour is in some way displeasing or perhaps surprising. In the US, particularly in the Midwest and South, it is a slang but not offensive noun meaning "small critter."
Hypothesis: Bugger is offensive to tall people.