I think we may have something missing here, an individual mechanism should not be making decisions about if authentication is mandatory, is there any additional reason why the mandatory check is being pulled into here?

In general Undertow will call each authentication mechanism in turn and if no mechanisms authenticate the request but authentication is required Undertow will take care of turning the call around and sending the challenged.