Original: Even the Sultan and his family were permitted entrance only once a year, on the 15th day of Ramadan, during the time when the palace was a residence.
Simplified: Even the Sultan and his family were permitted entrance only once a year, on the 15th day of Ramadan, during the time when the Palace was a residence.