The book title derives from the Gospel of Matthew (23:26), as quoted early in the novel: "Woe unto you, scribes and Pharisees, hypocrites! For ye make clean the outside of the cup and of the platter, but within they are full of extortion and excess."
John Hodder is recruited from a small town and congregation somewhere in the East to become rector of the august St. John's Church in a large midwestern city (modeled on St. Louis). Originally built in a rich area of the city, the surrounding area of the church including now-infamous "Dalton Street" has fallen into decay. But the church patrons include the wealthiest of the city, who travel to church on Sunday from their new mansions on the west side of the city, and who desire an "orthodox" minister who will preach a Christianity which will not question the growth of corporate power and the corrupting influence of concentrating wealth in America. Hodder develops a close friendship with Eldon Parr, the most wealthy and powerful man in the church. This is remarkable in that Parr has few close friends. Hodder comes to know of Parr's intense loneliness and unhappiness, being widowed and estranged from his son and daughter.
Hodder's failure to move his congregation causes him a spiritual crisis. He meets Josiah Bentley, one of founders of the church who left when financially devastated by Parr, but who has developed a network for helping the poor in the neighborhood surrounding the church. Hodder eventually disregards his "orthodox" views and preaches what he understands to be the core teachings of Jesus - to love and serve mankind. This spawns a huge battle between the old guard at the church and those who are deeply touched by Hodder's message, including Parr's daughter Alison. Though deprived of his salary by the vestry, the bishop is heartened by Hodder's transformation and tells him he will not recommend him for an ecclesiastical trial for heresy.

Question: Why did Josiah Bentley leave the church?
Answer: He was no longer wealthy because Parr destroyed him financially.
Alternative answer: lost his money