Thankfully if you miss your chance to recruit Eligor in the first palace he can be found later in the Path to Chemdah, Mementos.