Even a free market requires defenses against theft, trespass, and coercion. In the marketplace of ideas, that means keeping the peace between the quiet and the loud, the retiring and the intrusive, the decent and the vulgar. It means maintaining some sort of control over the noisy and the nosy.
Free speech is a right shared by two individuals simultaneously.
Mutual consent is the key. Consent spells the difference between nudity in the bedroom and in a public park, between pillow talk and dirty words on the public radio, between courtship and sexual harassment. But half a loaf of consent is much easier to find than a whole. Every right to speak collides with some reciprocal right not to listen, not to speak, not to watch, not to give away your own words, thoughts, or privacy. And so, to maintain a fair balance between those who wish to watch or listen or speak and those who do not wish to be watched or heard or bothered, to protect the public against perverts who like to inflame and psychotics who react violently to inflammatory expression, even the most liberal governments have always relied on the city hall to license parades, on the courts to enforce copyrights, on employment commissions to prevent sexual harassment, on public censors to suppress indecency, hate, and incitement, on the police to suppress fighting words, on the state to protect the softly spoken from the loud and the retiring from the nosy.