Only add site-lisp directory if it exists.