The truth is that many speakers are trying that now and are failing miserably. The sales cycles for unknown speakers have stretched so far that one could starve without alternatives. If I give a good speech, then the client will hire me again, and so will the decision makers. The truth is in the second sentence, the first sentence being an incorrect assumption. You will not reach that kind of relationship if you just focus on speaking and not use your other resources to benefit the client.
Speakers talk about their background, how funny they are first, etc.
The underlying issue here is credibility, which can be obtained without becoming an expert in a particular industry. Many speakers have provided that credibility by becoming industry experts. The nonindustry nichers do cross industry lines. They niche in different ways by size of company, and structure, or by culture. And, there is a credible argument about putting all your eggs in one basket.