Since government does not generate wealth, but simply spends wealth generated by the people, the only costs being borne are those costs borne by the public. When the public gets part of its own money back, that costs the country nothing. What we should really be concerned about are the costs created by government. Not all these costs are paid for in taxes. Some are paid for in many other ways.
Housing is a classic example.
If a builder were so optimistic as to imagine that he could build a 20-story apartment building in Silicon Valley, he would quickly discover that he would be lucky if the powers that be allowed him to build something half that height. Not all costs are financial. The other half of the people who rent apartments in Silicon Valley will find the prices driven sky-high by the artificially restricted supply and the growing demand. Open space, like height restriction, has its benefits. So do millions of other things.