I think I know this, because even though I haven’t personally graduated from so much as high school, a lot of my friends and family are academics.

Let’s start with your career as a doctoral student. If you personally find great interest in your research, but nobody else does, then you will find it very hard to get published (except in bullshit fake conferences and fake journals that accept any author who pays the fee) and you will not be able to persuade your committee to accept your dissertation, so you will not get a doctorate and will not graduate. Also, if no PIs at your institution think your research is interesting, you will not get funded as an RA, and you will have to devote a lot of your time to teaching instead of research, or you will have to pay tuition. If your research is in a field that requires access to equipment (e.g. chemicals, supercomputers, nuclear reactors, oscilloscopes, pencils) then not getting funded as an RA also likely means you don’t get access to the university’s equipment, so you have to buy your own, which may completely stop your research or just slightly annoy you, depending. If your research involves experimentation on animals or humans, the IRB or AEC is not going to approve your application just because you personally find great interest in the research; you have to convince them that it’s worth the potential harm to the experimental subjects. Finally, and perhaps worst of all, if you personally find great interest in your research, but nobody else does, then only your ideas are going to go into it; nobody else will contribute, and so you will progress maybe 1% as fast as if you were surrounded by a community of like-minded peers.

But suppose you work on stuff your advisor likes until you graduate in seven years, and then another four years in postdocs, and then you publish a bunch of papers as an assistant professor (on subjects that referees find so interesting that they recommend publication) and lots of other people find your ideas so interesting that they cite your papers so much that you are eligible for tenure, and then you actually get tenure because you didn’t grab any students’ asses or fail to actually physically be present when you were suppposed to be teaching or anything like that. So now, after 7+4+6=17 years of working on ideas that other people thought were interesting, you have tenure and you can, if you so choose, work only on things that you personally find great interest in.

At least in theory, and modulo the concerns about IRBs and AECs above. But your department is going to be a lot happier with you if you are working on things that you can find funding for, and if your ideas are attracting brilliant applicants to the department’s doctoral program, so that they can then admit those applicants and coerce them into working on the ideas they (your colleagues) find great interest in, in exchange for a lottery ticket that might pay off with tenure. In practice, though, tenured faculty tend to spend a great deal of time writing grant applications to persuade grantmakers that they will work on ideas that the grantmakers find interesting, giving talks to persuade their colleagues to be interested in their work, and teaching classes with at least the ostensible objective of persuading students to be interested in their work. Also, sometimes you have to ask grantmakers and grantmaker-like agencies not only for money but also for permission; nuclear research in the US, for example, requires that your research reactor be overseen by the DoE.

So, those are some of the structural incentives in the US academic system to work on things that other people in your field think are potentially big ideas, instead of just things you personally find great interest in. You know all this bullshit about how the web is an “attention economy”? Academics have been desperately struggling for attention for centuries.

And, yes, you need to be interested in what you’re investigating, too; not least because, if you aren’t, you’re going to have a hell of a time getting anybody else to pay attention, and you almost certainly aren’t going to make it through grad school.