The Crusaders used it as a palace and a church, but El-Aksa became a mosque once more after Saladin conquered Jerusalem.
El-Aksa changed hand a couple times in the course of history.