Abstract

1. Introduction
Various Internet websites allow reviewing movies and performances. These websites can be regarded as a resource for improving the quality of creative works so long as the information presented can be statistically analyzed at a reasonable cost. In this context, automatic review mining and summarization has been gaining attention from various fields, such as product and service businesses. Zhuang, et. al. (2006) focus on movie reviews, and find that reviews have unique characteristics. Specifically, movie reviews address not only movie elements such as screenplay, but also movie-related roles such as directors, screenwriters, and actors. They decompose review mining and summarization into 1) identifying characteristic and opinion words, 2) determining the class of feature word and its opinion polarity, 3) for each feature word, identifying relevant opinion words and obtaining valid feature opinion pairs, and 4) summarizing of the discovered information. Chaovalit, Pimwadee, and Lina Zhou (2005) investigated movie review mining by two approaches: machine learning and semantic orientation. Mohammadi, Gelareh, Sagae, Vinciarelli, and Morency (2013) explored the effect of multi-modality and perceived personality on persuasiveness of social multimedia content by using movie review clips from YouTube. Their interest is similar to ours in that they tried to clarify factors affecting degree of persuasion. We present a method for analyzing the determinants of ratings in reviewing creative works by using the case of movies in Japan. In Japan, the Yahoo Movie Review website presents viewer ratings, comments, the number of followers, and evaluation by followers. As a first step, we developed a system of automatically extracting information from the website, and generating an Excel-format database. This system was developed in the PHP programming language after analyzing the HTML format of pages on the website. In the second step, we developed a system for extracting information from this website and analyzing keywords in comments for positive and negative emotional expressions by using the "IBM SPSS Text Analytics for Survey" text-mining software. This allows conversion of text data to numerical data, and subsequent application of principal factor analysis and regression analysis. We then generated variables representing evaluations of "plot," "cultural value," "main theme," and "setting" of each movie, and analyzed the directional effect of each variable on rating by regression analysis. This paper presents a case showing positive relations for plot, cultural value, and setting, but a negative relation for theme. More elaborate development of the analysis method is expected to facilitate detailed clarification of factors that stimulate viewers' interest.
2. Method
2.1 Characteristics of the movie review site In Japan, "Yahoo Movie Review" is a well-known movie review site. The characteristics of this site are summarized as follows. 1. Free-description-type comments Reviews include text comments containing various emotional expressions that can be linked with various types of words, including nouns. Evaluation scale 2. Reviewers evaluate movies on a scale from 1 (lowest) to 5 (highest). Degree of review usefulness evaluations 3. Reviews themselves can be graded by readers, especially regarding review usefulness. Posting date and time information is available 4. Posting date and time information are used in adjusting the number of followers, because the number increases as time passes. We use reviews of the movie "Ask This of Rikyu ( Rikyu ni Tazuneyo )" for analysis. The address of the review site is http://info.movies.yahoo.co.jp/detail/tymy/id345694/ . This is a historical drama based on a novel by Kenichi Yamamoto, for which he won the Naoki Prize. The main theme of the movie is the "aesthetic" pursued by an historic tea master Sen no Rikyu. A controversial plot point is that Rikyu was inspired by the aesthetics of a Korean woman who was kidnapped by the Japanese. It is a historical fact that Rikyu loved tea cups and various items related to tea ceremony imported from Korea. However, many Japanese people are reluctant to accept this fact, and believe that the tea ceremony originated in Japan. For this reason, both extremely positive and extremely negative reviews are presented. An example negative review follows. The presented origin of the tea ceremony is unacceptable, delivered in a manner like a Korean drama, which is quite sloppy. In the movie, Rikyu explores aesthetics, but its source is a tragic love of his youth. The movie proclaims that this aesthetic, which originates in Korean culture, overwhelms the power of authority, and thus cannot be transmitted during Rikyu's life. Rikyu committed ritual suicide to protest the actions of Hideyoshi Toyotomi [a preeminent daimyo, warrior, general, and politician of the Sengoku period]. The movie suggests that the true reason for his suicide is a mystery. I think Rikyu acts as a foil to the complexity of Hideyoshi because of his origin as a farmer, and he could not understand aesthetics fundamentally. This invoked hatred between Hideyoshi and Rikyu. Rikyu said "I obey only aesthetics," and it is natural that authority could not tolerate these words. Ebizo [a famous kabuki actor] exhibited a certain aura while playing Rikyu, but I think his charm is not as fully exhibited as it is in kabuki. We investigate factors that determine the final evaluation of reviews by examining the review text and evaluation points given by reviewers. This task is important because impressions of reviews are biased by highly emotional words, making it hard to examine the actual factors that determine the evaluation. For example, words such as "disappointed," "nonsensical," "absurd," "below the level of B-movie quality," "regret seeing it," and "contempt for Japanese aesthetics and history" are observed in the reviews. Without analysis, such descriptions may make readers believe that the movie is not worth viewing. It is not obvious, however, how assigned movie ratings and ratings of the reviews themselves are affected by these words. Analysis allows examining the actual factors that affect grading and clarifies the true functioning of movie review sites.
2.2 Text mining
We used "IBM Text Analytic Survey" as text mining software. The first step was to extract words, and the second was to categorize those words. When categorizing words, emotional words were used to link nouns and verbs. By selecting the type of category, we can depict linkage among words. Here, we selected positive emotional words and extracted the related words. Figure 1. Screen of IBM Text Analytic Survey As is shown in Figure 1, the text mining software relates words with emotional words such as "good," "bad," "praise," and "grief." The number of words relating to these emotional types is listed in the bottom-left window of the software. By selecting, for example, "good" and "praise," we can produce a bar chart of the frequency of related words in the upper right window. We then exported this output as an Excel file, combined it with the original Excel file, and produced the SPSS file.
2.3 Extracting factors that determine evaluation grading
The next task is to extract factors that determine evaluation grading. We apply principal factor analysis to reduce the number of variables and classify the words. As a result of the analysis, we extracted four factors for words associated with positive emotional words.
Tables 3 and 4 present the results of principal factor analysis for words associated with negative emotional words.
Table 4. Structural matrix of principal factor analysis for words associated with negative emotional
3. Regression analysis on the determination of factors for grading We conducted regression analysis with the factors derived by principal factor analysis on the set of words associated with either positive or negative emotional words. Table 5 and Figure 2 show the results for grading evaluation. The standardized coefficients are comparable in size because the variable unit is normalized. Larger values of the standardized coefficient imply that the effect on grading evaluation is stronger. The results show that "Tea Ceremony" was associated with positive emotional words, and had the strongest positive effect on evaluation grading. "Rikyu" was associated with negative emotional words, and had the second strongest positive effect on evaluation grading. "Nationalism" and "history" associated with negative emotional words, and decreased evaluation grading. "Korea" was associated with negative emotional words, and increased evaluation grading.
Table 6 and Figure 3 show the results for grading usefulness. "Tea items" was associated with positive emotional words and had a positive effect on grading usefulness. "Tea ceremony" was associated with positive emotional words, but had the strongest negative effect on grading usefulness. "Nationalism" and "history" were associated with negative emotional words and had a strong positive effect on grading usefulness.

Relation between evaluation grade and sales
4.
To understand the role of movie reviews in more detail, we examined factor determinants that explain sales. Data were taken from the website of the Motion Picture Producers Association of Japan ( http://www.eiren.org/toukei/index.html ). Table 7 shows the results of regression analysis on sales determinants. The number of evaluators is strongly associated with sales, but evaluation rankings had no explanatory power. Toho and Toei are the two largest Japanese movie companies, but as Fig. 4 shows, GAGA achieves the largest average sales per movie. These results suggest that high evaluations do not guarantee the commercial success of a movie. Commercial success is, however, reflected in the number of reviewers. It is interesting that Toei films have higher average evaluations than those from Toho, but the average sales and number of reviewers are smaller. In particular, compared with Toho films, Toei films had far fewer reviewers. One possible explanation is that Toho targets mass-market consumers, but Toei produces movies targeting film connoisseurs.

5. Conclusion
The results of regression analysis show the merits and the limitations of this analytical procedure. This analysis allows separation of factors influencing reviews and those that do not. Without this kind of analysis, it is difficult to distinguish whether factors associated with emotional words actually affect grading, even when the emotional words give a strong impression to review readers. One limitation of this analysis is ambiguity in interpreting the results. "Tea ceremony" was associated with positive emotional words and had a positive effect on grading evaluation. The result itself gives limited interpretable information. To interpret the results, it is therefore necessary to defer to the original comments. Alongside such follow-up tasks, the information given by this kind of analysis provides us with precious information regarding movie reviews. Finally, we would like to discuss utilization of review sites by movie producers. We showed that commercial success is not guaranteed by review evaluations. However, this does not imply that the grade does not reflect movie quality. The method we propose allows judging the true factors in determining evaluation grading and gives us detailed information on the relation between quality and evaluation. In the movie market there exists a stratification of viewer quality, where mass-market viewers can be considered of medium or low quality. Commercial success in the mass market does not necessarily guarantee long-term success in the movie business without improvement to movie quality.