The primary challenges of using glass in architecture are its vulnerability to damage and breakage, as well as the difficulty of cleaning and maintaining its transparency and functionality.