Plus Yoko had caused trouble for the group by trying to have other band members. And members of the general public. Someone was going to have to pull the pin on our general malaise. Looking back on it, it was a tough situation. When you're all mates and someone wants to jump ship for an entirely plausible reason, it seems strangely plausible to take it as being an implausible act.
Emotions get in the way.
As does the pot, the wine and the shagging of anything whatsover. And then there's the music. You can always get a new guitarist or bassist if they both leave, but you can't always 'acquire' a vibe. If you don't have intimacy with each other, you lack something. Losing that intimacy is like leaving a relationship.