It is more than just declining government budgets which have caused the decline. In my opinion, the biggest impediment to corporate R&D is the modern emphasis on quarterly earnings and meeting third-party analysts' estimations thereof.

(Warning: idle speculation and arm-chair philosophy ahead)

When executive compensation is tied primarily to short term results, it seems inevitable that "most" companies will end up being run by those with the most talent and agressive behavior in this area. Those people are usually bean counters. Every company needs a bean counter, but only in the last (say) 25 years has it been the norm for every company to be LEAD by one.

It is that last point where I think the problem manifests the most. There is, as everyone at HN knows, a difference between management and leadership. Leadership requires vision, and no matter how visionary or innovative you are, if your ultimate goal is having a good day at this quarter's earnings call, it's hard to see how you could ever justify investment in something more than a few quarters off.

I'm greatly simplifying, of course. I don't think this happens the way the naive view of this problem proposes: that productive, storied R&D departments are summarily executed at the hands of a new CEO hellbent on meeting his numbers (well, I'm sure that DOES happen sometimes). Rather, I think what has probably happened to most corporate research is this:

- The company is doing well, they're meeting their numbers every quarter and everything is peachy. Much lip-service is paid to the R&D team "without whom none of our success would be possible".

- Something happens. Usually it's a bad economy or a disastrous product launch. Maybe it's a huge corporate scandal. Whatever it is, it's worse than just a few quarters of missed estimates. The stockholders are screaming; the analysts and their puppy dog journalists are holding pitchforks. The CEO promises that something will be done. Decisive action will be taken to bring this mess under control.

- At first, the R&D departments are spared, because no one wants to "destroy the golden egg" (or insert some predictable seed-and-flower metaphor). But the first round of layoffs in middle management, manufacturing, and engineering aren't enough. The stock price is still suffering. Clearly more drastic action is necessary.

- But wait a minute, we hear in the boardroom, what about this thing they're doing in R&D that's costing $30 million a year and probably won't produce anything marketable for 20 years, if at all? I mean, R&D is vital, but surely that stuff is better left to universities and foundations, right? And just like that, the farthest-looking research project is cut.

- It's easy to see how things can proceed in this manner, with every downturn or crisis prompting a cut in the next most distant research project, until suddenly the R&D department is just the "Two to Five Year D" department. At that point, they get renamed to "Product Centers", and are truly and officially subordinated to the management and politics of the quarterly earnings cycle.

I spend so much time laying this out because, just as these forward-looking places didn't disappear overnight by some nefarious CEO twisting his or her mustache, so can't they be made to magically reappear at existing corporations through some courageous magnanimous gesture of a new superhero CEO.

If you're the new CEO of Yahoo! (a pretty young company when talking about stuff like this), it's probably possible. But "Verizon Labs"? "P&G Long Term Research"? I don't think so. In fact, I predict that even the storied Microsoft Research division will encounter this fate if the company's financial prospects continue to look south.

So what will resurrect the tradition of corporate R&D? I think several things:

1. Just as there is a trend toward online distributed learning, so I think will there emerge a trend toward online distributed research. The Open Source movement is, in a way, the archetype for this (although it is closer to distributed engineering). With new peer-to-peer open publishing and review systems (of which arXiv is a start), a lessened immediate emphasis on PhD credentials, a desire to mix and match areas of interest out of fun and curiosity, the continually shrinking barrier to entry (with everything from 3D printing to cheaper genomics), and the always-important promise of possible fortunes through startup spinoffs or "acqui-hires", it will become more and more common for fundamental research to happen outside the traditional places where it was done before.

2. The larger, more expensive research will still be done at universities and companies, but the companies will be more of the Elon Musk variety. In fact, I very much hope that he succeeds with Tesla and SolarCity as well as SpaceX, not only because I believe in those underlying visions, but because I think Wall Street and the rest of the world need to see "altrusitic visionary capitalists". Skepticism and cynicism of capitalism in-the-large is at an all-time high, and it will take some larger-than-life personalities with spectacular successes to get those less courageous people and organizations to follow suit. The ultimate end goal of this should be, among other things, "the death of the quarterly analyst" and, hopefully, a deepening of trust in the mechanisms of well-regulated capitalism supported (especially in the technical realms) by open and public-funded foundational research at universities and other institutions.

3. Long term, I really believe in the elimination of most physical scarcity. If you can 3D-print a Ferrari, you might not care as much about stepping on so many toes to get one. This is, I admit, a hopelessly naive and optimistic look at the future, and to some extent human beings will always compete with one another (it is, in my view, one of the things that make life worth living). But in a world of post-physical scarcity, things like fundamental research and truly innovative product and service development may become THE things to brag about "at the club".

One can hope.