Some Libertarians are also pacifists and decline the use of any force. Libertarianism is broad enough to encompass pacifists. All oppose the initiation of force. Libertarianism is broad enough to encompass militants. The common factor is the opposition to the initiation of force.
Freedom is the absence of the initiation of force.
The robber and bully have initiated force. The condition of freedom doesn't exist unless there is an absence of the initiation of force. You have the right to earn a living, but not to compel others to provide your living. Libertarians apply the Non-Coercion Principle to ALL human behavior. It doesn't matter if the initiators of force are in or out of government.