Remove clang_library from the config

Currently this breaks on my mac, so I've removed it until I find another
solution that handles my other linux boxes and mac.