Create a small build matrix with optional libraries disabled