When Luffy and Zoro referred to “they,” they were likely talking about the crew members of Blackbeard (also known as BB).