This is exactly what Helen Yuhasova had to do in 1946. When she couldn't locate any other interested women, she formed The Solitary Cyclist of Washington D.C. and met at unspecified intervals with herself. Most groups do contain more than two people. In fact most contain more people than their name implies. The Fifth Northumberland Fusiliers, however, have never quite equalled the full strength of that illustrious regiment.
Some groups start with a chance meeting, two people discovering a common interest in Holmes.
A teacher's question to an elementary school class led to the founding of The Shadows of the Elms. Sometimes it begins with an ad in a newspaper or magazine. The Scowrers began in Anthony Boucher's column. The mystery is bothering Clifton Andrew of 400 Water Street. Thus began the Scandalous Bohemians of Akron, the first society to admit women.