Detailed Instructions: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Problem:We consider the following metrics for automatic evaluation of different submissions. Joint goal accuracy has been used as the primary metric to rank the submissions.

Active Intent Accuracy: The fraction of user turns for which the active intent has been correctly predicted.

Requested Slot F1: The macro-averaged F1 score for requested slots over all eligible turns. Turns with no requested slots in ground truth and predictions are skipped.

Average Goal Accuracy: For each turn, we predict a single value for each slot present in the dialogue state. This is the average accuracy of predicting the value of a slot correctly.

Joint Goal Accuracy: This is the average accuracy of predicting all slot assignments for a given service in a turn correctly.
Question: How are the models evaluated?
Solution:
Active Intent Accuracy Requested Slot F1 Average Goal Accuracy Joint Goal Accuracy