I've spent my career at early stage companies, sometimes as CEO sometimes as CTO and many times as an engineer.

It may be right that you become trained to be a Startup CEO by being a Startup CEO, because there are some things that you cannot get any other way. However, you can get a lot of the necessary perspective as a Startup Employee and then Startup Founder. This way you will start with a big leg up compared to a CEO who has no startup experience. I think probably it would be a good best practice for us entrepreneurs when starting our career to start as employees, then on our second job be founders, then if you want the third company would be the one where you become CEO. Thus when you are founding a company, look for someone who has previously been a founder and an employee, to be your CEO. I'm not saying you need a grey hair, but someone who has been on the roller coaster before and who knows what he is signing up for. First job out of the gate as CEO is romanticized by the likes of Mark Zuckerberg et al. but it's not the best chance for your company. Generally.

Also, critical for a Startup CEO is that the CEO have training in the primary industry of the company.

IF you're disrupting real estate sales, and your primary competitive advantage is legal or structural (eg: an old school not really technical business) the CEO needs to be a salesman. In this way the CEO is experienced in the primary area that determines success.

If you're doing this with software, however, the CEO needs to be an engineer (not a salesman.) As an engineer who has learned to sell (by being a sales engineer, after creating an evangelism department which became our marking department) learning to sell was rather easy, and a CEO as salesman can work with pure salesmen very well, with the CEO coming in as the Closer.

But a CEO without an engineering background who is trying to manage a company whose primary business is software development is going to be a problem-- the only way this works is if you have a co-founder who is an engineer and is effectively in total control of the technology side.

Maybe a better characterization is that, sales is a skill that can be taught. Certainly talented sales guys have a talent that can't be taught, but anyone can be taught to be a competent sales guy. Engineering does involve skills that can be taught, but that capacity for grasping engineering concepts, that seems to not be something you can teach. Either that or I've just seen a lot of non-engineering CEOs who didn't care to learn enough about engineering not to undermine it.

The number one killer of startups in my experience is arrogance and the Dunning Kruger effect (they are linked.) VCs forcing bad pivots on companies because some MBA with 2 years experience as a junior associate at a VC firm thinks the company (run by industry veterans with 20 years of experience) should chase the latest fad for example (and yes I've seen exactly that though most cases are less obviously asinine.)

But I've also seen the CEO who thinks because he's CEO he HAS to "Make Decisions" or he won't have "authority" undermining the technology side he knows nothing about... but is too arrogant to realize he's the cause of the companies problems. That I've seen a whole lot too[1].

A lot of the founder fights I've seen have been due to this too-- the CEO keeps pushing the CTO to do something wrong, that the CTO knows is wrong.

So, the real thing I would look for in a CEO of a startup is that they are the kind of person who is not focused on their own ego, not arrogant.

In fact, I'm starting to think that the CEO role is itself obsolete.

First off, there's three people in the company and one of them's title is "Chief executive officer"? Three words none of which actually involve getting things done?

I think a better arrangement might be one that is flatter where at the top of the org is a committee made up of the heads of the departments who have to reach consensus, with the Chairman of the Board being the tie breaking vote... but it should never come down to votes. There should be someone who is on that committee who is understood to have the best product vision and who has the loyalty of the rest of the committee regarding the product vision. That could be the chairman (in which case he acts like CEOs do now, only less involved in the day to day stuff where he may be tempted to meddle.)

I'm not completely sold on this idea, just thinking about it. At any case, the totally hierarchical 5 layers of management in a 50 person company structure is out of date by about 50 years I think.

[1]Example of the above: Stopping engineering from using an issue tracking system (which he was not interested in logging into even though it built burndown charts and predicted ship dates and gave him great visibility) and forcing everything to go onto a single medium sized white board with sticky notes. (which are constantly falling off!) So, instead of QA writing up an issue with reproduction steps, we are working of of vague notes on sticky notes, like "server disconnects from peer", or "memory is too much". This has dramatically slowed us down, probably, along with other process undermining decisions, cost us a good 2 months in the past 5 of delay, etc. But of course its engineerings fault for not meeting the (arbitrary) deadlines he picked. (Yes, I'm a little bitter, it sucks to see a potentially great company tanked by this kind of stuff. It's salt in the wound to see the business guys running around arrogant and making comments about how bad engineering is, when we can't even get them to give ups a spec, let alone participate in an agile process.)