I think this kind of thing (I choose to call it stage fear) has a life of its own. I describe some phases of it in my life:

Absence of it:
Up to some point in your life its not there at all. I remember in our school, we used to have a teacher's day. I which teachers would take rest, and students from higher classes would go and teach junior classes. Per student you were required to take 3 classes. And I remember going to all three of them, totally free and un-encumbered. And did that with flying colors, in my opinion. At that stage in life, I didn't know what stage fear was, as it was totally absent. I call it the innocent phase.

Onset and very acute straightaway:
When I was in college. I came from a background where we had good written English, but didn't have fluency in speaking. And I think that made me very self conscious in my new environment. There were occasional hiccups in conversation in English, in a very friendly setting, but I felt very embarrassed. And although everyone is rather kind, but it led to some people slotting me, and they would choose to talk to me in Hindi, the next time, and it was all in good intent, without malice. But you can imagine, what kind of effect it had on me. I began dreading things like any project presentations in front of the class. Stopped asking questions in the class. And of course camouflaged all of this very well - by trying to carry a non-caring air. Hey I was a 20 year old then.

Working on it, and it being there in a low intensity way:
This phase carried on to my work life. But the intensity was less. But this was a good phase to work on this problem. I tried to reason myself out of this. I told myself things like: "Why do I need to fear people? They are just here to listen to me. Something which I can offer them. And if I am sincere and honest and humble, then what do I need to be afraid of? They won't hit me?" . Another thing which helped me do well is thinking that, I will start the presentation, by confessing my extreme nervousness, and request the audience to bear with me. Surprisingly, this relaxed me a great deal. I never had to do it, as I felt better just by thinking this.

Another thing, which helped me was prayer. I feel in a more equanimous state by the act of praying - and I have made suitable modifications in my thought that I am conversing with nature, instead of God. (I don't believe in a lot of aspects of religion. But also believe and have experienced solace by it. Agnostic. Full disclosure.)

A dormant demon?:
Talking about now. I don't feel that intense kind of anxiety now, in speaking. And thankfully so. I think Startup life also helps in that regard. As there are hardly any BS presentations, or needing to say things you don't believe in. If you have to speak to VC, its to get them to invest. If you speak to an audience, you want to describe your idea, which you believe in. And also being honest, I felt, helped a lot. As it puts you in the right frame of mind. So of course nervousness is there, but much less. I was lucky to speak in front of a rather large audience in the early stages of this Startup journey, and I did not feel as anxious.
But once you have experienced, this kind of anxiety. You can never be too sure. It can surface, at any unprepared moment, in the middle of an answer, when you grow self-conscious. I guess the simple key is to acknowledge it, and try to reason it out. Even if you stutter or stammer. Apologize with a smile. Take a deep breath. Think. And speak to the human (or humans) who only want to know what you think. Cheers!

Edit: typos