Complex sentence: The only other visitors to the area were the ones who came to the ancient Hanuman Temple, who came from the walled city on Tuesdays and Saturdays and that too before the sunset as the passage back wasn't considered safe during those days.
Split into simpler sentences: The only other visitors to the area were the ones who came to the ancient Hanuman Temple. Such visitors came from the walled city only on Tuesdays and Saturdays and before sunset, as the passage back wasn't considered safe during those days.