## 2 Materials and Methods
### Data Annotation Process  
Two linguists with expertise in Indonesian language and sentiment analysis manually annotated the dataset. Annotation focused on four predefined aspects: teaching quality, curriculum relevance, infrastructure, and services. Sentiments were categorized as positive, negative, or neutral. The inter-rater reliability was assessed using Krippendorff's Alpha, yielding a near-perfect agreement (\(\alpha>0.99\)). Automated labeling was employed for the remaining dataset using a pre-trained sentiment analysis model, ensuring consistency with the manually annotated subset. The aspects analyzed teaching quality, curriculum, infrastructure, and services were selected based on a systematic review of existing literature on higher education evaluations and a preliminary dataset analysis. These aspects represent core dimensions frequently highlighted in student feedback and are critical for assessing higher education institutions' quality of learning experiences.  
The aspects were defined as follows:  
1. Teaching Quality: Students perceive lecturers' effectiveness, clarity, and engagement during teaching activities.
2. Curriculum: Evaluations of course content's relevance, coherence, and comprehensiveness in meeting academic and industry standards.
3. Infrastructure: Assessments of the adequacy and accessibility of campus facilities, such as classrooms, laboratories, and internet access.
4. Services: Feedback on administrative and support services, such as academic counseling, registration, and other institutional services.  
The final set of aspects and their definitions were refined iteratively through consultations with subject matter experts and pilot annotations of sample data. These definitions served as the foundation for the annotation guidelines.  
The annotation was conducted by two annotators with expertise in Indonesian language and literature. Both annotators hold advanced degrees in linguistics and have prior experience in text annotation for sentiment analysis. Before commencing the annotation process, they participated in a calibration session to familiarize themselves with the guidelines and ensure a uniform understanding of the task. The guidelines provided explicit instructions for identifying the predefined aspects and categorizing sentiments as positive or negative. Annotators were also trained to handle variations in linguistic expressions, including informal and idiomatic phrases commonly found in Indonesian student reviews.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Aspect** & \begin{tabular}{c} **Positive** \\ **Sentiment** \\ \end{tabular} & \begin{tabular}{c} **Negative** \\ **Sentiment** \\ \end{tabular} & \
\begin{tabular}{c} **Total** \\ \end{tabular} &
\begin{tabular}{c} **Aspect** \\ \end{tabular} \\ \hline Teaching & 1239 & 1261 & 2500 & Teaching \\ Curriculum & 1257 & 1243 & 2500 & Curriculum \\ Infrastructure & 1259 & 1241 & 2500 & Infrastructure \\ Services & 1263 & 1237 & 2500 & Services \\ Total & 5018 & 4982 & 10,000 & Total \\ \hline \hline \end{tabular}
\end{table}
Table 4: Distribution of reviews by evaluated aspect.  
Both annotators annotated an independent subset of the dataset to evaluate inter-rater reliability. The agreement between annotators was assessed using [PERSON]'s Alpha. The results of this analysis are summarized in Table 5 demonstrating [PERSON]'s Alpha value of 0.99998 (\(\alpha\) > 0.99). This value indicates near-perfect agreement between the two raters, confirming the high consistency and reliability of the labeled dataset. Discrepancies during the initial annotation phase were resolved through discussion, leading to further refinement of the guidelines and ensuring uniformity throughout the entire dataset.