## 1 Introduction
Specifically, the feedback rating matrix is factorized into user and item latent matrices, and the factorized matrices are multiplied to build an estimated feedback rating matrix that does not have unknown ratings [12]. Figure 1: Co-occurrence network of keywords constructed by analyzing negative review texts (ordinal ratings under 3) collected from a manuscript editing service. The size of a node and the thickness of an edge respectively represent the frequency of a keyword and the co-occurrence frequency of the two connected keywords. However, MF usually suffers from a data sparsity problem [13; 14; 15], which arises when the number of ordinal ratings is small and the ratings are concentrated to certain users or items. The problem is even worse for the ordinal ratings in manuscript editing services as the number of manuscripts that a researcher has written is usually small and not all manuscripts receive an editing service. Moreover, a huge portion of users do not leave their feedback in the form of ordinal ratings. Several attempts were made to address the data sparsity problem of ordinal ratings by additionally using indirect user feedback such as review texts and behavior logs. For example, [PERSON] et al. [16] and [PERSON] et al. [17] utilized review texts for a car recommendation and item recommendation, respectively, and [PERSON] et al. [18] adopted users' visit frequencies for a location recommendation. However, extracting user opinions from the indirect feedback is usually costly and difficult as it requires a number of complex processes such as data collection and preprocessing [19]. On the other hand, there are a few studies that tried to utilize a type of direct user feedback called binary ratings to supplement ordinal ratings. Binary ratings are an indication of a user's opinion in a binary form, whether a user liked the service or not. Previous research reported that the binary ratings provide intuitive and accurate information about a user's opinion [20; 21; 22]. Moreover, binary ratings are relatively easy to collect compared to other types of user feedback, such as ordinal ratings and review texts, since users can express their opinions by simply clicking a button--like or dislike [23]. For this reason, binary ratings were exploited with ordinal ratings simultaneously to infer a user's opinion in previous studies. [PERSON] and [PERSON] [24] proposed a factorization method which finds a latent matrix that has information depending on both ordinal and binary ratings and tried to estimate unknown feedback ratings by utilizing the matrix. In addition, [PERSON] et al. [22] suggested transfer by mixed factorization (TMF) to incorporate binary ratings with ordinal ratings by adding an extra component to the conventional MF. Most manuscript editing services collect both ordinal and binary ratings in order to monitor service quality and provide a function for users to favor or exclude a certain editing expert. Figure 2 shows screenshots of two web pages, where users can leave their feedback on the received services. A user can leave a binary rating--like or dislike--for an editing expert on the page shown in Figure 2. When a user selects the like button for an expert, the expert will be assigned priority for future services. In addition, a user can give an ordinal rating with a review text to an expert at the page shown in Figure 2, which can be reached by clicking 'leave feedback' button in Figure 2.  
Figure 2: Screenshots of two web pages, where users leave their feedback, (**a**) binary ratings, (**b**) ordinal ratings and review texts, on the received editing services. While exploring user feedback logs collected from a manuscript editing service, which are composed of ordinal ratings, binary ratings and review texts, we were able to observe the limitations of ordinal ratings on expressing negative opinions. Figure 3a,b show boxplots representing the distributions of sentiment scores (\(x\)-axis) of review texts according to the ordinal and binary ratings (\(y\)-axis), respectively.