I'm afraid not. Swipe to remove the conversation, but everyone else will remain in the group. When someone sends a new message to the group, you'll receive a notification, and the conversation will reappear in your messaging app.