## 4 Dataset Annotation and Filtering
### Annotation Process
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Annotating Tool** & **Advantages** & **Disadvantages** \\ \hline \multirow{4}{*}{**Brat**} & **Open Source:** Brat is an open-source tool and free to use for annotation tasks. & **Lacks Machine Learning Integration:** Brat cannot integrate machine learning models or perform active learning. \\  & **Web Server Deployment:** Bart can be deployed on a web server, which provides a safe and regulated setting for group annotation. & **Initialization Difficulties:** Compared with some other tools, it is harder to set up the configuration files. \\  & **Customization:** Gives the user freedom to define entity types and annotation policies. & **Limited Features:** For intricate or cooperative annotation tasks, NER Annotator might not provide all the functionality you need. \\ \cline{2-3}  & **User-Friendly:** An easy-to-use tool developed for non-technical users with a friendly User Interface design. & **Limited Features:** For intricate or cooperative annotation tasks, NER Annotator might not provide all the functionality you need. \\  & **Efficency:** A simple and effective tool for NER annotation tasks. & **Lacks Active Learning:** Neither machine learning model integration nor active learning is included in the NER Annotator tool. \\ \hline \multirow{4}{*}{**Tagtog**} & **Collaboration Features:** Provides sophisticated tools for collaboration, enabling several annotators to collaborate on a single project. & **Paid Service:** A paid membership may be needed for some services, which makes it less affordable for large-scale projects with tight budgets. \\  & **Data Preparation:** Reduces the process of preparing data and can aid in the training of machine learning models. & **Complexity:** Advanced features need technical experience to utilize. \\ \hline \multirow{4}{*}{**Prodigy**} & **Active Learning:** Utilizes active learning strategies that enable the creation of highly accurate annotation models quickly and with less effort. & **Cost:** Prodigy is a paid tool, which makes it not suitable for some applications. \\  & **Scalability:** It is suitable for small-scale and large-scale annotation applications. & **Complexity:** For users who are unfamiliar with machine learning integration, utilizing this tool may be difficult. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Annotation tools comparison: features and considerations. **ORG (Organization)**: Identifying associations and communities mentioned in the text, such as companies and institutions. * **WORK-OF-ART (Work of Art)**: Recognizing literary works and manuscripts within the text, such as books' titles and poems' titles. 4. **Entity Correction and Alignment**: After using DeepPavlov to annotate the text, reviewing and editing the annotated entities is a crucial step that should be performed. To make sure they are precisely aligned with the matching text spans, we closely inspected the annotated entities. The entity boundaries are checked for accuracy and contextual relevance, and adjustments are made as necessary. 5. **Regular Expression Annotator for \"WORK-OF-ART\" Entities**: To compensate for the potential poor performance of the DeepPavlov pre-trained NER model, a unique regular expression annotator is constructed for the \"WORK-OF-ART\" entity. This guarantees the proper recognition and annotation of [PERSON]'s literary works inside the text, which might vary greatly and rely on context. 6. **Verification by Domain Specialists**: Domain experts carry out a thorough verification procedure on the final annotated dataset. These specialists examine the annotated entities concerning the documents' context to make sure that the annotations appropriately capture the semantic meaning of the entities and are consistent with the underlying material. This step involves identifying and fixing any inconsistencies or errors.