There are plenty of systems other than those two. Most of the others have been better than either when tried. A few were worse. And quite a lot of the very worst of the worst were utterly indistinguishable from libertarian capitalism in all but name. Feudalism, for one. 
The thing is libertarianism doesn't even work in theory. If you really think about it, and think further than you are led, and consider empirical fact then the inevitable conclusion is that it can only lead to feudalism... If you think just a little further you realize that was the intent all along. 

There are far too many logical contradictions, flagrant fallacies and blatant impossibilities in it for that many people to have all failed to notice. Praxeology is not real logic it's what you use when you want to make bovine manure look like a reasonable facsimile of logic. 
Honest  philosophies use formal logic to protect against fuzzy thinking.