> This is a cliche.

It's not, really. At least, not in the sense that it doesn't actually happen. What GP describes is part of the sociolinguistic phenomenon of “register” [1], wherein people have different little context-dependent “dialects” or “styles” that they use in different situations: for example, you might use a different sublanguage when speaking at home than you'd use at work, and even that one might be different from one you'd use in a job interview, or giving a TED talk, or writing a paper for CACM, vel sim. A particularly well-known, if extreme, example would be “babytalk”. The obverse of babytalk,—uh—“smartypantstalk”, seems to be a perennial literary criticism throughout history, sometimes famously so (Mark Twain (v. James Fenimore Cooper) and Ernest Hemingway (v. William Faulkner) come to mind).

> There are speakers that just have a vocabulary perceived as complex and it's not any easier for those people to switch to "easier" words than it is for others to switch to more complex words.

Although registers and register-switching are a natural part of language (and people switch between registers all the time without even being aware of it), I think it also true that different people have different registers for the same situations. So, perhaps your casual register seems to many of your speech partners more like their own formal register, or otherwise more like what they expect a formal register to seem like. Or maybe you were raised in an environment where the formal register was perfectly appropriate for use in casual contexts, and so it actually is your formal register (which just so happens to also be your casual register).

In fact, I think it might be harder to reduce your vocabulary than to build it, since in adding words you need not consider any particular speech partner or their understanding, but in removing words you must consider many possible speech partners and whatever you might know of their vocabularies, to make effective decisions about which words need removed. I imagine it involves a lot of trial and error.

> I think all speakers of non-standard traditions (complex, simple, dialect, etc.) have to deal with a lot of prejudice.

This is certainly true, and one of the main topics of sociolinguistics. The particulars of our speech as perceived by our listeners are used to infer a deal of paralinguistic (and usually social) information, right or wrong (factually or morally). This is one of the reasons for register-switching: if I'd written this comment in the same register I'd use to talk to my 8-year-old nephew, I suspect you'd come away feeling rather confused and put-off (or you might infer that I'm mentally handicapped); on the other hand, I doubt my nephew would understand much of this comment as-is. By register-switching, I'm able to optimize my language not only for an appropriate level of linguistic competence, but also for an appropriate social setting, depending on how I want to be perceived, and how I want my listeners to judge my own perceptions of themselves. In some sense it's tactical, but it's a mostly subconscious and automatic operation, instinctive and reflexive, a natural part of language.

I think, however, there's something worth keeping in mind: depending on the situation or context, speakers of even standard dialects can be subject to prejudices. This is more seldom and, in most cases, less consequential (I think) than what speakers of non-standard dialects have to deal with, but, myself being a native speaker of “General American”, it's something I've experienced first-hand. I think that, perhaps, it has less to do with “standardness” per se than with the prestige associated with the dialect—it just happens that standard dialects are also usually prestigious. At the same time, I think the prejudices come from usual in-group vs. out-group dynamics, where your dialect can be a pretty distinct marker of your membership status. (I'm not much of a sociologist, however).

> ...and for some anecdote: For some reason the way my family talks is rather formal and I often have to deal with people assuming that I'm very pretentious, which really isn't ideal in most cases. (I'm working on it!)

I know this is difficult— Good luck! :)

--

[1]: https://en.wikipedia.org/wiki/Register_(sociolinguistics)