## 5 Results
### Zero-Shot Topic Classification
#### 5.1.3 New Results on DailyDialog--Topic and Subtopic Level  
In view of the previous results, a final test was performed using the manual annotations made in [29] on the same 1000 random turns selected from the DailyDialog dataset. For the cases where the label was \"other\", manual annotations were again performed, but this time taking into account the 18 proposed topics (Table 1), instead of only the original 13 topics. The results of the annotations are as follows (numbers in parentheses indicate frequency of occurrence in the dataset): food (184), finance (119), fashion (108), vehicle (56), family (39), sports (27), education (18), music (15), art (13), animals (12), weather (7), movies (6), books (4), photography (4), video games (3), news (3), science (3), and the bag-like category other (379). The generated topics were then compared with manually annotated topics. For the subtopics, an expert performed new manual annotations. For these annotations, only the 579 turns where the BART and manual annotations matched at the topic level were taken. These turns were then annotated at the subtopic level. During annotation, only the subtopics corresponding to the annotated topic were shown to the annotator. Table 7 shows some examples of the sentences annotated in the DailyDialog subset. The results in Table 8 at the topic level show an F1 score of 0.45 (57.90% accuracy), representing 579 out of 1000 turns manually classified by humans and by the BART classifier. This is a reasonable result, given that the BART model is capable of classifying up to 18 different topics (Table 1), and given the large number of different topics from which the classifier can choose, causing the error to rise. The confusion matrix is shown in Figure 6.  
The results are similar to those obtained for DailyDialog in Section 5.1.1, but in this case, there are 18 topics (Table 1) versus the original 13 topics; hence, there may be a decrease in the F1 Score. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Sentence** & **Toipic** & **Subtopic** \\ \hline Human1: & I know. I’m starting a new diet the day after tomorrow. & food & healthy \\ Human2: & It’s about time. & & \\ \hline Human1: & I have something important to do, can you fast the speed? & vehicle & speed \\ Human2: & Sure, I’ll try my best. Here we are. & & \\ \hline Human1: & Do you know this song? & music & song \\ Human2: & Yes, I like it very much. & & \\ \hline Human1: & Where are you going to find one? & other & — \\ Human2: & I have no idea. & & \\ \hline Human1: & I wish to buy a diamond ring. & & \\ Human2: & How many carats diamond do you want? & finance & investment \\ \hline Human1: & It’s a kitty. & & \\ Human2: & Oh, [PERSON]. I told you. No pets. It’ll make a mess of this house. & animals & pets \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Examples of sentences and their topic and subtopic annotations. The teal color represents Human 1 and the violet color represents Human 2. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model** & **Accuracy (\%)** & **F1 Score** & **Precision** & **Recall** \\ \hline Topics & 57.90 & 0.45 & 0.43 & 0.53 \\ Subtopics & 88.95 & 0.67 & 0.70 & 0.68 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: DailyDialog human annotations vs. BART classification. The analysis of the confusion matrix demonstrates a significant level of consensus across almost all topics. Notably, the topics of animals, books, family, fashion, finance, food, music, other, photography, sports, vehicle, and weather exhibit a higher frequency of correctly identified turns. Conversely, there are relatively fewer instances of correctly identified turns within the domains of art, education, movies, news, science, and video games. This higher percentage of successful identifications, in comparison to previously obtained results, can be attributed to the incorporation of a more refined level of granularity in the topic classification.