Don't tell them what you're going to do. If Joe Wise-Arse ruins it, proceed with a different trick, they won't know.

There is a 1940 booklet called "Outs" Precautions and Challenges by Charles H. Hopkins, that is devoted to precisely this problem. As I recall it is just as suitable for close-up as stage magic. It is a playing cards only book, but the principles remain valid for all magic.

There was a film industry guy, forget his name, who insisted that if someone wants to see you do some tricks (informally) you must first make them prove that it matters. Have the host set up a room or an area specifically for you to do you magic. Then require anyone who wants to see your impromptu act come to you. It's about setting a tone.

Just being very experienced and performing flawlessly sends a message as well.

Others like Slydini, simply did not perform close-up magic unless conditions were perfect, and it was very obvious that the magic was wanted. He had a presence, and a lifetime of reading audiences. When he performed, he was in charge, no one doubted it. However, Slydini, like Daryl, had tricks tailored for perfect conditions, and tricks tailored for the worst of conditions.

Some like John Carney, student of Dai Vernon for many years, are just so darn natural, that you don't realize they're doing magic until your jaw hits the table. So with John, there's simply not much chance of interrupting, because it just seems like your having a little chat. He is the master of doing things on the off beat. He often seems genuinely surprised that something magical happened. Of course his knowledge of outs is extensive.

The late lamented Daryl was not afraid to show a little bit of anger if someone kept interrupting. Nothing rude, but the offender got the message. Not everyone can do this. Daryl was amazing in his planning. When he wrote up a trick, he described exactly what conditions it was suitable for: seated, 3/4 surrounded; seated 180-degs surrounded; standing, audience in front; standing surrounded, etc. Methods differed for the same trick depending on conditions. Not every trick could be performed under every circumstance, and he noted this when describing them. He used some pretty advanced techniques known to some close-up workers to control the angles in less than favorable circumstances.

Ricky Jay, would get extremely angry if you interrupted his trick, or demanded he shuffle the cards or such. He'd lecture you with the ruined deck sitting in plain view, and remind you that you had agreed not to mess with his performance. After a minute of anger (acting?) and hectoring, he'd give in, take the deck you'd apparently ruined by shuffling, and proceed to astound you with its successful conclusion. But then, Ricky Jay was a student of magic, from ancient to the modern, that has had very few peers in that respect... perhaps Vernon, Mulholland, Marlo, Diaconis, and a few others.

Bottom line, without polish, brashness, and flawless execution, most of us will get our tricks ruined by our audience now and then. Just go the confidence store and buy a couple of large bags of the stuff. Real magic is hard work, study, and confidence. Can't help you with the last one. I feel your pain. Voice of experience.