Calculus is an elegant subject that is not nearly as difficult to pick up as its reputation leads people to believe. But there are two main problems with learning calculus today.

One is that mathematics in general is taught very poorly (well, all subjects really, but mathematics bears the brunt even more so). There's a great concentration on "teaching to the test", rote memorization, "plug and chug", churn through as much "material" as possible and then forget it over summer, that sort of thing. Cargo cult education, basically. This is very damaging to the teaching of calculus because it is very much a subject that is dependent on some very fundamental core concepts. If you don't "get" those core concepts, then calculus will be forever opaque to you. Worse, this pattern of "education" applies all the way down the line to every math subject taught in school, including the pre-requisites for calculus: advanced algebra and trigonometry.

And calculus is very dependent on actually knowing that pre-requisite math. Not being dependent on plug-and-chug, guess-and-check, "eliminate 2 wrong answers from the multiple choice question and then guess", or "cram for the test and then forget it". All of those techniques work well at gaming the educational system, but they leave you weak in the actual skills of applying trigonometry and advanced algebra to problems, and using them as tools while learning other, more advanced, mathematics. So you have a typical calculus course trying to lay down enormous concrete slabs of new concepts and "material" but it just sinks into the bog rather than building up a structure sitting on a solid foundation.

Additionally, calculus is often used as a filter course. Calculus has a potentially infinite amount of "trivia" associated with it. All manner of rules, substitutions, identities, definitions, theorems, and so forth. As such, calculus effectively has a "difficulty dial" than any teacher can set arbitrarily depending on how much memorization and busy work they want their students to slog through. And many teachers decide to set this at just high enough of a level to make calculus a real pain in the ass for most students. There's always a million justifications for doing this, but most of them are pretty weak. There's a justification for acquiring familiarity with all of these things, and proving you can walk through the steps to do harder problems, but in practice it's mostly useless. People who actually use calculus day to day generally just keep a cheat sheet around of integration and derivation rules and substitutions and whatnot, or they employ the use of a computer to do the heavy lifting. Once you understand the concepts there's not much value in slogging through many mindless examples just for bragging rights.

All of this leads to people getting the impression that Calculus requires superhuman skills to learn (no, it just requires being actually caught up with the pre-requisites before hand) and also that the application of calculus tends to be boring and monotonous. If education did a better job of ensuring the pre-calculus fundamentals were thoroughly in place, and concentrated more on concepts and applications of calculus (like physics, engineering, geometry, control systems, computer graphics, and so much more) would, I think, lead to not only a lot more students succeeding at learning calculus well but also enjoying it.