tt ††† 700–8530 3–1–1 † 3–1–1 700–8530 †† , ††† E-mail: †pldu28hq@s.okayama-u.ac.jp, ††niitsuma@suri.cs.okayama-u.ac.jp, †††ohta@de.cs.okayama-u.ac.jp Paragraph Vector . Paragraph Vector , ScoreSent2Vec , Paragraph Vector . ScoreSent2Vec . ScoreSent2Vec , Paragraph Vector ScoreSent2Vec , ScoreSent2Vec word2vec, 1 Amazon.com .com   ² , Mikolov word2vec [1] [2] . word2vec . ScoreSent2Vec , Paragraph Vector . word2vec ScoreSent2Vec Vector ,  Brown  Corpus [6] [5]. 1 news, religion, hobbies, science fiction, romance, humor 6 can, could, may, might, must, will 6 Skip-gram (CBOW) [1] [2] Continuous  Bag-of-Words , TF-IDF Bag-of- Paragraph Vector 0.487 0.494 . 2 , ScoreSent2Vec Words (BOW) F-measure bin , Paragraph 4 , ScoreSent2Vec 0.609 . word2vec Vector 0.584 , word2vec Paragraph Vector [3] ScoreSent2Vec [5] [4] BOW , Paragraph Vector [3]. , Paragraph Vector [5] ScoreSent2Vec , ScoreSent2Vec http://uuu.amazon.com 1 http://kakaku.com 2
i . 1. 5 1 5 . , Paragraph 2 . , . , 2. ScoreSent2Vec . word2vec Paragraph Vector word2vec , Paragraph Vector word2vec Mikolov 1 Skip-gram king – man + woman (1) king, man, woman man king woman .  Mikolov , word2vec queen , Athens – Greece + Norway Oslo word2vec [2]. word2vec    , w(t − c), ..., w(t − 1), w(t + 1), ..., w(t + c) .  , Mikolov word2vec word2vec . Twitter Paragraph Vector, ScoreSent2Vec , Paragraph Vector ScoreSent2Vec . 0.847   , 6 2.1 word2vec Mikolov word2vec Skip-gram 0.746 2.2   Paragraph  Vector Paragraph Vector word2vec CBOW [1] [2]. 2 . Paragraph Vector word2vec .  Skip-gram 2 Paragraph Vector with Distributed Bag of Words PV-DBOW CBOW Paragraph Vector with Distributed Memory  PV-DM 2 Skip-gram :  Skip-gram w(t) .  w(t − c), ..., w(t − 1), w(t + 1), ..., w(t + c) PV-DBOW :  word2vec Skip-gram Skip-gram CBOW CBOW 2 . Skip-gram . word2vec Skip-gram Skip-gram w(t)出力層2 , 2 (1) .入力層2 CBOW (1) [7] [2]. , 6 ,                ,         , F-measure . C . . , , , 2,     4 .  w(t − 2), w(t − 1), w(t + 1), w(t + 2) w(t) ,  , w(t − 2), w(t − 1), w(t + 1), w(t + 2) , w(t) 2 w(t − 2), w(t − 1), w(t + 1), w(t + 2)     , word2vec 3   PV-DBOW Paragraph Vector . 2.4 ScoreSent2Vec Paragraph Vector ScoreSent2Vec [5] . ScoreSent2Vec      , Paragraph Vector Paragraph Vector score vector          . ScoreSent2Vec . PV-DBOW 2 Scored Paragraph Vector with Distributed Bag-of-Words Scored SPV-DBOW Paragraph Vector with Distributed Memory   SPV-DM 2 . . ID , word2vec SPV-DBOW Paragraph Vector    PV- DBOW , score vector .  PV-DBOW :      4      PV-DM PV-DM CBOW , word2vec . .  PV-DBOW CBOW word2vec ID ID .  score vector word2vec SPV-DM :       6      SPV-DM , Paragraph Vector    PV-DM . , score vector Paragrh  Vector [8]   , Vector Paragraph SPV-DBOW (1998-99 ) (1998 ) , ScoreSent2Vec 61.2  , [9] graph Vector 77.9 https://github.com/niitsuma/ScoreSent2Vec Para- 3. 10 , PV-DM PV-CBOW(Continuous Bag-of-Words of Paragraph Vectors) F-measuer 200 0.9431 2.3  Paragraph Vector   word2vec Paragraph Vector    word2vec Paragraph Vector         word2vec , , ScoreSent2Vec . score vector出力層. 4 4   PV-DM ID , . .  PV-DM , . .出力層
ScoreSent2Vec , scale . word2vec , python gen- . Paragraph Vector sim   ³ klb3713 sentence2vec    4 . ScoreSent2Vec [5] 4.1 SSTD SSTD . SSTD , Amazon Mechanical Turk  5 .  Amazon Mechanical Turk
Very Negative, Negative, Neutral, Positive, Very Positive 5
8,544 , 1しParagraph Vector . Paragraph Vector . score vector . ScoreSent2Vec Paragraph Vector, Concatenate, ScoreSent2Vec Paragraph Vector . Concatenate score vector Paragraph Vector d-size 100 6 106 , score vector , ScoreSent2Vec 6 3 . 1 6 can, could, actor, may, might, must, will, 1 cast, film, scene, sound, story, 1 4 5
3 score l (2)
4. 2 . 1 . , Stanford Sentiment Treebank Dataset [10] (SSTD) Positive Negative , Brown Corpus religion 1 news . 3 d-size . min-count . . scale . score vecor
, L . P(l,v) vector score vector ν   , .
(2) , V
but, good, great, nice, no, not score vector , AIC, BIC . AIC, BIC (2) LassoLarsIC BIC LassoLarsIC Very Negative, 5 0, 1, 2, news, religion , news, religion, hobbies, science fiction, romance, humor 16,964 6 5 . scikit-learn [11] , AIC . Negative, Neutral, Positive, Very Positive 3, 4 4.1 3 Paragraph Vector, Concatenate, ScoreSent2Vec score vector . 4.1 ScoreSent2Vec 6 w . 1 , 6 5W1H, what, , can, could, may, might, must, will, 1 when, where, who, why, how, 1 back, even, first, last, long, new . , 5W1H  , 6 , AIC, BIC 3 (2) BIC 4.1 .       , AIC LassoLarsIC LassoLarsIC humor, hobbies, news, romance, religion, science fiction 0, 1, 2, 3, 4, 5 0, 1, 2, 3, 4, 5 AIC, BIC Logistic Regression scikit-learn . Logistic Regression , d-size, min-count, scale 9 4 . Paragrph Vector, ScoreSent2Vec 4.2   Brown Corpus Brown Corpus Logistic Regression .  Brown Corpus 16,964 , 1961 . Brown Corpus 2 . , 6 score vector ), ), ）） (3) 2 3 , 3 3 3 , . 5 ,
—— Paragraph Vector   — 7 ──助動詞群　──5W1H群　──頻出単語群8 d-size , d-size, min-count, scale    3 1 .  . 7                 5W1H d-size , 2 9 min-count ──助動詞群　──5W1H群　──頻出単語群10 scale .  , ScoreSent2Vec            Paragraph Vector, Concatenate .  , AIC, BIC . . ,          , AIC , , , BIC , AIC 4.2 . .  d-size 3 , 5W1H   , .  ,                                              , AIC     5W1H 8,     9,      10    , d-size       , min-count        , scale ScoreSent2Vec . ,     10 5. , .  . 5.1 3 , AIC, BIC              , . 3         2 . 10,754 . 4.1 , AIC, BIC BIC .      , AIC, BIC . ,  16,964 . 1 . , 2 , , . . , AIC, 1 news religion Very Negative Negative, Very Positive Positive , Neutral BIC ScoreSent2Vec [1] [2] [3] [4] Mikolov
. min-count , min-count , min-count
6. , scale
ScoreSent2Vec 2
, d-size
, min-count 10 , scale 0.01 AIC, BIC 5.2 ScoreSent2Vec . 4.2 8 , d-size d-size, min-count, scale Mikolov, T., Sutskever, I., Chen, K., Corrado G. and Dean, J.: Distributed representations of words and phrases and their compositionality, Advances in Neural Information Processing Systems , pp. 3111–3119 (2013). Mikolov, T., Chen, K., Corrado G. and Dean, J.: Efficient estimation of word representations in vector space, arXiv preprint arXiv:1301.3781 , pp. 1–12 (2013). Le, Q. and Mikolov, T.:  Distributed Representations of Sentences and Documents, CoRR , abs/1405.4053, pp. 1–9. (2014). Socher, R., Perelygin, A., Wu, J. Y., Chuang, J., Man- ningand, C. D., Ng, A. Y. and Potts, C.: Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank, Proceedings of the 2013 Conference on Empir- ical Methods in Natural Language Processing , Stroudsburg, PA, Association for Computational Linguistics, pp. 1631– 1642 (2013). :  Paragrph  Vector , Vol. 2015-DBS-162, No. 11, pp. 1-8, 2015. Francis, W. N., Kucera, H.:  Brown Corpus Manual, Brown University (1979). : Word2Vec 21 , pp. 441–444. [6] [7] (2015). [8] [9] [10] [11] SSTD Brown Corpus 17,000 77 , pp. 165–166 (2015). Socher, R., Perelygin, A., Wu, J. Y., Chuang, J., Manning, C. D., Ng, A. Y., Potts, C.: Recursive deep models for se- mantic compositionality over a sentiment treebank, Confer- ence on Empirical Methods in Natural Language Process- ing, pp. 1–12 (2013). Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, É.:Scikit-learn: Machine Learning in Python, The Journal of Machine Learning Research, Vol. 12, pp. 2825–2830 (2011). . d-size , d-size 10
. ScoreSent2Vec : , . NL, pp. 159–166 (2004). :  Paragraph  Vector . , Vol. 162, 50 100 [5] 1