David commanded to gather together the foreigners who were in the land of Israel; and he set masons to hew worked stones to build the house of God.
Allowed, tolerated, used by God Himself, yes; but it still stands as a thing David conceived, promoted, commanded, planned, prepared for and projected in its entirety.  From its conception in the mind of David, and throughout its entire history, the temple, in a sense, similar to pagan temples all over the world, was the conception and project of a mortal human being, not of God.