Your going to hate my answer because it makes the problem worse.

For about two weeks I was convinced my new roommate was a hallucination and I'd finally snapped. There were a number of uncanny cases where he knew the same incredibly obscure trivia and references that I knew. Things like "trap remixes of musicians that sold their soul". That's too specific to write off as just similar interests by men in a similar demographic. There were also a number of uncanny instances of not knowing the same exact things I didn't know. I had never seen him outside of the apartment. We had no mutual friends. We were both in the same unusual living arrangement, permanent temporaries at an Airbnb partially bartering code / home automation work as barter for rent. At some point the topic of eye color came up, and someone pointed out how all of us in the room have green eyes, and how that is the least common eye color and 3 of us is very statistically unlikely. Eventually I had to ask myself which is more probable, there really are two people with all these characteristics in common that ended up at the same place by coincidence, or I'm having a schizophrenic break and this is my delusion? My possibly imaginary friend had mentioned that he lived in Russia through the first grade and speaks Russian at a first grade level. I do not speak Russian. So I ask him to teach me some Russian grammar. He agrees but then changes the topic. I ask him to teach me some Russian. He says sure but avoids the question again. I ask him to teach me a bit of Russian. He agrees and evades again. At this point I am having some very serious doubts about my grip on reality.

Eventually we do meet other people from each other's circles. After a while there's been enough mutual third parties acknowledging both of us, that either a very large cast of characters are my delusion or this man is real. I can never actually prove one way or the other, but the scales are now tipped towards real by all the people standing on them.

And so we get to the answer to your question. "Reality is shared consensus." There's a shared consensus that my roommate was real, and that consensus may as well be reality because I can't distinguish it from the case where everyone has the same delusion.

Of course reality isn't the shared consensus per se. Reality is not subject to a referendum. However, shared consensus is the multimeter we use to read reality. Where is the difference between a 9 volt battery and "if I touch the proved to the terminal it reads 9 volts"? There isn't one.

In your example, my definition of "reality is shared consensus" becomes a big problem. If there are two groups with their own consensus then those are two realities. You are free to believe anything and everything is true right up to the point of fatally erroneous belief. For things where the consequences of being wrong are not so sure, there is nothing forcing a consensus around "objective reality".

Niels Bohr allegedly had a horse shoe in his office. When asked he said it was for good luck. One audacious visitor asked "do you really believe that?" He replies "no, but they say it works even if you don't believe in it." This was cheeky of Bohr. Plenty of people don't believe in quantum mechanics, but they say it works anyway. If someone truly insists on believing magic horse shoe theory and rejecting quantum mechanics, there is nothing that will force them to acquiesce to our objectively correct answer.