Here we should throw an exception instead of using the default metrics -- we want the user to notice this