Default Error handling should be with custom error page as defined in

The call to JError::setErrorHandling(E_ERROR, 'message', array('JError', 'customErrorPage')); found at the start of includes/framework si wrong, should be 'callback' instead of 'message' so that 404s are handled by the system default error page.
This is usually not visible by users, because the Redirect system plugin eventually calls JError::customErrorPage($error);

However, if that plugin is disabled, the the 404 error page complety breaks, displaying warning and notices, and no 404 error message at all

It appears problem was the same in J 2.5