Teaching ESL without a textbook is known as Dogme ELT. It's certainly possible to do this and many teachers have done so successfully. They employ a variety of other activities and games and focus on what the students are curious about and need to learn.