They decline the use of any force. Libertarianism is broad enough to encompass pacifists. All oppose the initiation of force. Some libertarians are militant. Libertarianism is broad enough to encompass militants.
Freedom is the absence of the initiation of force.
The robber and the bully have initiated force and the condition of freedom doesn't exist unless there is an absence of the initiation of force. You have the right to free speech, but not to compel others to provide your forum. You have the right to earn a living, but not to compel others to provide your living. You have the right to believe in whatever religion you choose, but if your god requires the sacrifice of virgins, you must find a virgin willing to be sacrificed without being coerced into it. Libertarians apply the non-coercion principle to all human behavior.