Revert "Add /etc/ld.so.conf so the dynamic loader has a path even if LD_LIBRARY_PATH is not set"

This reverts commit dca513882cec49ae570605f3eace49fc63dfc357.

Because that did not do it either. We have to fix ld-linux