## 3 Designing a Big Data Evaluator Recommendation System
### Structure of the Evaluator Recommendation System  
As shown in Figure 1, the big data evaluator recommendation system starts with the collection of documents received from the project submission system. The project submission system in the form of the relational database (RDB) collects both the information related to the project submission (such as submission year, project number, principal investigator, project name, and research budget) and the original text of the submitted project. The collection includes copies of the same data in the old system to prevent the performance of the actual service provision system from being compromised and preserving the original copies (to prevent deformity due to refining). The original collected texts are processed and analyzed first through the original text processing module. The data for different projects are processed and saved as files so they can be used on the big data platform. The process also indicates where in the documents, the data are extracted from, such as the subject, summary, and body, so that different scores can be awarded to the same keywords and weighted later. Subsequently, the keyword extracting module performs sentence breakdown, such as the analysis of morphemes, by processing the data. In this way, keywords are extracted, and the weights for different keywords are scored using the term frequency-inverse document frequency (TF-IDF) algorithm. Keywords extracted in this way are stored in the research keyword knowledge database and are accumulated as basic data that can improve the data quality by sorting synonyms and unused words, e.g., The core of the big data recommendation system is recommendation quality, so quality improvement activities should be continuously conducted. Various information are stored in the research keyword knowledge database in the form of an ontology for efficient quality improvement activities in the future. Figure 1: Original text processing module. The collection of data on the research achievements of the evaluator candidates is conducted using a method similar to that of the project application data collection. The data from the original text files, primarily in hwp, doc, and pdf format, are extracted in the same manner in the text processing module, and the keyword extraction module is indexed in the search engine for comparative analysis with the project application data. Unlike in the case of project applications, the data on the research achievements of the evaluator candidate are collected regularly, and preliminary indexing reflects changes only in the modified copies through daily placement. The system in its early phase requires the initial indexing of all researchers. As the data on all researchers cannot be complete, priority is placed on key evaluators whose problems due to omitted data can be minimized. By comparing the keyword values extracted from project applications and the index extracted from the research achievements of the evaluator candidates, the project comparison module searches, scores, and ranks tachievements similar to the project applications. The similarity scoring module calculates the overall similarity scores for evaluator candidates based on the scores that the project comparison module creates for different achievements. These scores are used as evidential scores to determine the most appropriate evaluator candidates for evaluating the specific project application. The original text data from the research project applications and papers, require preliminary work through the original text processing module to extract keywords. The most common formats for the research project applications and papers are hwp, doc, and pdf, which require varying text extraction modes. Doc and hwp files can be extracted through open libraries and do not involve significant errors. However, pdfs face many issues because data, such as line breaks and encoding, may be lost due to conversions from text file formats such as doc or hwp.