Fix conformance_test.rb to load libraries properly

Previously the files installed into system is loaded if exists