Police say scores of people managed to swim to safety , but many more may have been swept downstream .