better handle indexation crashes + use long timeout and delay for links generation until optimization