They can claim it is not really a free market because force is involved. But then they cannot explain how the finer granularity they propose for their own system will stably solve this problem. At an ultimate level, the force must be there to determine ownership. They can try to explain the lack as a market failure, but this conflicts with their rhetoric about market failures being due to interference within state controlled markets. If they ignore that rhetoric, and demand some sort of regulation to produce the results they want, we then get to chide them for impractical inconsistency, and ask why we shouldn't regulate other markets as well.
Some libertarians protest that they have never signed the contract.
But our society has long recognized the validity of implied contracts. For instance, eating dinner in a restaurant obligates you to pay for it, even though you haven't signed anything. Children are another example of the implicit nature of the contract. Many libertarians argue that because children enter a social contract by accident of birth, it's not fair that they should be subject to a contract they never agreed to. But there is no accident here.