It can be difficult to identify a competent leader as you pointed out. Here are some of the things I've learned to look for:

1. They don't tell they show.

CEO 1 told me his company has a culture of honesty and transparency.

CEO 2 told me, "This is an ambitious project. We've only got 18 months left of runway if we keep hiring at the rate we need to in order to get the product to market. We're confident that if we launch this year we'll be profitable enough to get further investment. It's high risk and high reward but I'm in the same boat you will be. If this doesn't work I'll be looking for a job too. I wouldn't be here if I didn't think we could do it."

CEO number 1 told me they were honest, CEO number 2 showed me.

And it's not just telling the truth, any statement they make about their company you should think about afterward and ask yourself what they did to demonstrate that statement.

2. Honesty deserves its own point.

When I interviewed at an internal startup of a Fortune 100 company the CTO told me, "The technology we're building is going to replace a lot of people at <parent company>. That means we have to be very careful about what we say and how we release things. I do my best to shield this team for the corporate politics above us but sometimes it slips through. They do pay the bills after all."

Every company has its benefits and drawbacks. It can be a legacy system that's difficult to work with, an industry that's heavily regulated, a bureaucratic organization, a codebase written by contractors, a talent shortage, or something else. They're hiring you to solve a problem. If they don't tell you what that problem is and the difficulty you'll face overcoming it then you should be highly skeptical.

If the job seems too good to be true it probably is.

3. They know where they add value and their limitations.

"How did I get my job? Basically, I was friends with the co-founder. Before this, I did market research at <large tech company>. I don't know enough about software development to build a tech org. That's what I want to hire you to do." - How one CEO started a conversation with me.

If you ever talk to an executive that thinks they're smarter than everyone else run like hell. Likewise ask what their day normally consists of (seriously, you'd be surprised by most answers).

The mentality that you want is that of a team. Everyone has their role to play and everyone they hired is a critical piece. I've found this also comes with a 'whatever it takes, just get things done mentality'. I've seen CTOs who personally refilled the office snack collection for months until they could hire an office manager.

4. They understand what skills are and are not transferable.

Ask them how they hire their managers and product owners. Look through the backgrounds of every executive at the company.

Equifax's Chief Information Security Office spent her entire career managing call centers. She had no background in either technology or security.

You might be tempted to say, 'so what management is a skill of its own.' True. But it's not a sufficient skill in and of itself. Hiring a manager for their management skills alone or a product owner just because of their experience being a product owner is like hiring a teacher just based on their experience teaching. Sure there are transferable skills every teacher has but you can't just throw that German History professor into a Nuclear Physics course and be like 'what it's all teaching'.

If there are any non-technical managers in charge of a technical worker's hard pass on the company. If any of the executives don't have a background in their field, hard pass. If they have product owners who are non-technical and have no background in the company's industry then get the fuck out.

5. Differentiate between managers and leaders.

Management and leadership are completely different skills. You want the latter.

Listen to how they tell the story of their company. Are they the center of attention or do they spend their time praising others? Ask how many people have left the company and why. Ask the lower level employees you interview with a question and see if the executive's answers match. Ask any technical manager when the last time they wrote a line of source code was.