I have experience as part of a hiring team that spent over a year searching for a single candidate, and I also have experience as a job seeker who has spent over a year searching for a minimally acceptable job.

In my experience, the only lesson I've learned is that you cannot know much at all about a candidate's eventual job performance or cultural fit from any hiring technique.

Giving them tests or quizzes doesn't work. Giving them take-home coding tests doesn't work. Asking them behavioral and technically probing questions about their background doesn't work. Having them come on-site and suffer through whiteboard hazing doesn't work. Making them sit in on a technical Skype call with three simultaneous interviewers doesn't work. Everybody's pet theory about how to vet them before their first day doesn't work. And most of those things also waste a lot of everyone's time and have the poor false-positive / false-negative rates that the OP describes.

The only thing that I think works is just hiring someone and then seeing if it works and not being afraid to fire them if it doesn't work. Everyone freaks out about this idea because there's so much taboo about the cost of hiring, setting up benefits, and so forth, and the morale drain from firing.

But it's a trade-off. Do you want to pay those costs while getting actual info about an employee? Or pay those costs in a status-showy way ("look how hard our hiring process is!") but not actually get any info for what you are paying?

To address the firing part of it, which of course is not at all fun, there is an easy solution: make generous severance packages a standard part of your job offers.

If you mistakenly hire someone who is not cut out for your team (for whatever reason, skill, attitude, work ethic, culture, etc.) that is on you no matter what your hiring process is. Don't make candidates bear the costs of your hiring mistakes. Treat them well and they will be more understanding if the hiring process involves a lot more firing than most are accustomed to. If you give them 6-8 months of salary as a severance benefit (and that's on the low end), then they will be secure about the risks of getting fired and it will buy them time to find a new job if you need to let them go. For a huge number of companies, this is easily affordable and building in a fixed cost of 1/2 year of their entry salary is borderline trivial (even if people want to make a big argument about it).

Everyone likes to whine that this is expensive, but hiring the right person is expensive -- it just depends on how you want to pay the cost. Like I said before, you can engage in a bunch of fake performative nonsense like tests and whiteboard hazing, in which case you are paying the price in terms of wasting your existing employees' time and in terms of false positives and false negatives and constantly leaving money-from-would-have-been-productivity on the table, or you can just own up to what it costs and be more willing to pay the dollar amounts necessary to trial people out. Then you can have much lower barriers to entry that mostly just involve basic conversations and extremely simple sanity checks about a person's background. If they pass those and the team more or less likes them, just pay the money to see if they work out and move on. Stop obsessing over it, especially by letting HR middlemen get in the way and obfuscate things.