It depends on a lot of factors. The preexisting relationship between your country and your destination is an essential factor. However, some other factors like mode of travel are taken into consideration. Generally, it is almost impossible to travel by air without a passport while it is possible by land or sea.