Education is guaranteed under article 26 of the Universal Declaration of Human Rights, which the US signed and helped author. Not providing a public option for education would mean redefining what we believe to be fundamental freedoms. How are you going to provide education to the poor without public schools? Will the government give the poor money for education? Will you subsidize the "private" schools with government money? That would defeat the whole purpose of having private schools as you aren't allowing them the operate in the free market. It's not clear 100% bastardized "private" schools would be any more competitive than the current hybrid system. Public schools and the belief that education is a fundamental human right are inextricably intertwined.

EDIT: I'm going to edit this comment to refuse the arguments below, as there are multiple posters with similar points of view.

The claim is that restoring "choice" to the educational market will increase the quality of education for all. This choice is already happening. Wealthy families choose to move to areas with high property taxes that fund excellent public schools. I grew up and was educated in such a neighborhood; when I was in HS the average salary of a teacher at my "public" school was $100k. Each year, students from inner-city Chicago try and register for the schools in my HS area. They are not allowed because the schools are full with residents. Therefore, they have to go the public school in the neighborhood they reside in.

In a private system, the same behavior happens, except the barrier to entry becomes price not the geographical location of your residence. Several people have suggested government vouchers to help defray the cost of education. Do you really think these vouchers will be able to buy a higher quality of education than what is current available? The best private schools will raise their price to the most efficient market point where they charge their students the most and enroll exactly enough students.

Now some might argue that we already have a private system by proxy. People are still paying for higher education, they are just doing it though home ownership. Why not just covert to a private system? The answer is this: the quality of the weakest school is far more important than the quality of the strongest school. People with money will always be able to find a way to get their kids an excellent education. As a society we don't need to worry about these kids. Society does need to worry about the bottom line. The long term cost of failed education to society is immense: increased crime rate which leads to prison costs, lower voter literacy which results in less qualified politicians, the list goes on. In a private system, the low end is forgotten about. The idea that a government voucher would be large enough for the bottom end is ridiculous. In the public system there is a constant struggle to push the low end higher. This is (part of the reason) why the public system costs so much. The public system has to bear an extra burden (for the good of society) that the private system eliminates through efficiency.

The bottom line: Choice is already taking place in the hybrid school system by proxy through property tax or expensive private schools. The 100% private system would be exactly the same, with the added disadvantage of a forgotten low end. Vouchers are just a half-assed replacement for the public part of the system already in place. If the vouchers were high enough to raise the low end to an acceptable level, the cost would be equal to the public part of the current system. In order to raise the low end of education above the fair market value, government regulation is by definition necessary. A small correction must be made for the failure in free market forces.