経営課題にAIを!ビジネスインフォマティクス研究会 (第13回)
JSAI Special Interest Group on Business Informatics (SIG-BI #13)

GPT-2

LSTM Language generation by GPT-2 and text analysis of economic fluctuations forecast
through LSTM1∗

1

1

Yoshihiro Nishi1, Aiko Suge1, and Hiroshi Takahashi1
11Graduate School of Business Administration, Keio University

Abstract: Economic fluctuations have a great impact on business management. In this study, we explore
the method to improve the accuracy of the prediction model of the economic fluctuation using the economic
watcher survey reported by the Cabinet Office, the Government of Japan. The economic watcher survey
includes interviewees' answers in the text and the economic business trend tag for each answer. The
classification was attempted through the LSTM model. As a result of the analysis, we found the prediction
accuracy could be improved by using texts generated by GPT-2. Further examination of the classification
model will be planned.

1
[1]
[2]
[3]
[1][2][3]
2000
[14]
[4]

[1]
[11]
[15]

3
2000

2
2,000

5

[5][6][10]∗:223 - 8526
E-mail: nishi_yoshihiro@keio.jp

× 4-1-1

経営課題にAIを!ビジネスインフォマティクス研究会 (第13回)
JSAI Special Interest Group on Business Informatics (SIG-BI #13)

[1]
2000
12019

Google Apps Script2019 955GPT-2
Google Apps Script

LanguageApp

244,3532

4.2 GPT-2
5Positive
Negative

51,000

2,000

GPT-2
40GB
35,000[12][13] Positive

GPT-2
24

Negative
4,441
52,333
117,290500
1,000

4.3 Word2vecWord2vecWord2vec[8]
Skip-gram

17,3542:

Positive

2,000

52,935244,353

CBOW[9]Word2vec Skip-gram

4.4 LSTM
LSTM (Long Short-Term Memory),RNN
LSTM1,000(Positive)[7]
1,000(Negative)

LSTM
Accuracy

2,000

4GPT-2
Word2vec1

LSTM

4.1 LanguageApp
LanguageApp1
1Web

Negativelength

1:

800

https://developers.google.com/apps-script/reference/language/language-app

1:

RNN

経営課題にAIを!ビジネスインフォマティクス研究会 (第13回)
JSAI Special Interest Group on Business Informatics (SIG-BI #13)

4:

5

1
12

LSTM

5.1
LanguageApp

2

1,800

3,600

200

400

2,000

4,000

5: LSTM

1”The number of units sold is
expected to increase as a result of new product launches.”

5.2

Accuracy

0.79

20.83

63GPT-2
Negative

1,000
2Positive
2,000

3:

12

Positive

1,000

2,000

Negative

1,000

2,000

2,000

4,000

[1],

[2]

:

[3]

[4]

Skip-gram
LSTM

scikit-learn

41
train_test_split

2

Accuracy

4Discussion Paper Series, No. 345, (2018):, In Proceedings of theThe 30th Annual Conference of the Japanese Society for
Artificial Intelligence, pp. 1-4, (2016)
Aishwarya A., Jiasen L., Stanislaw A., Margaret M. C.,
Lawrence Z., Dhruv B., Devi P.: VQA: Visual Question
Answering, In Proceedings of the International Conference
on Computer Vision, (2015)[5]Fung G. P. C., Yu J. X., Lam W.: Stock Prediction:
Integrating Text Mining Approach using Real-time News,In Proceedings of the IEEE International Conference on
Computational Intelligence for Financial Engineering, pp.
395-402, (2003)[6]

Gidófalvi G.: Using News Articles to Predict Stock Price
Movements, Department of Computer Science andEngineering, Technical Report University of California,(2001)
Hochreiter S., Schmidhuber J.: Long Short-Term Memory,

0.1
5LSTM
2――,
, No. 18, (2018), ESRI

5.3
Word2vec:1[7]

経営課題にAIを!ビジネスインフォマティクス研究会 (第13回)
JSAI Special Interest Group on Business Informatics (SIG-BI #13)

Neural Computation, Vol. 9, No. 8, pp. 1735-1780, (1997)[8] Mikolov T., Chen K., Corrado G., and Dean J.: Efficient
Estimation of Word Representations in Vector Space, In
Proceedings of the International Conference on Learning
Representations Workshop, (2013)[9] Mikolov T., Sutskever I., Chen K., Corrado G., and DeanJ.: Distributed Representations of Words and Phrases andtheir Compositionality, In Proceedings of the NeurIPS,(2013)[10] Mittermayer M. A.: Forecasting Intraday Stock Price
Trends with Text Mining Techniques, In Proceedings of the37th Hawaii International Conference on System Sciences,(2004)[11] Nishi Y., Suge A., Takahashi H.: Text Analysis on the Stock
Market thorough "Fake" News Generated by GPT-2, In
Proceedings of the INFORMS Annual Meeting, (2019) (toappear)[12] Radford A., Narasimhan K., Salimans T., and Sutskever I.:
Improving Language Understanding by Generative PreTraining, Technical Report OpenAI, (2018)[13] Radford A., Wu J., Child R., Luan D., Amodei, D., andSutskever I.: Language Models are Unsupervised
Multitask Learners, Technical Report OpenAI, (2019)[14] Reiter E., Dale R.: Building Natural Language Generation
Systems, Cambridge University Press, (2000)[15] Zhang R., Guo J., Fan Y., Lan Y., Xu J., Cheng X.:
Learning to Control the Specificity in Neural Response
Generation, In Proceedings of the 56th Annual Meeting of
the Association for Computational Linguistics, pp. 11081117, (2018)