7th Asia-Pacific Conference on Plasma Physics, 12-17 Nov, 2023 at Port Messe Nagoya

Prediction of the plasma vertical instabilities using BERT
Sejung Jang, Hiroaki Tsutsui
Tokyo Institute of Technology, Tokyo, Japanjang.sejung@torus.nr.titech.ac.jp
Achieving and maintaining the conditions necessary for
nuclear fusion is a complex and challenging task. One ofthe primary challenges faced is effectively suppressing
plasma instabilities and disruptions. In this study, we
employ Bidirectional Encoder Representations fromTransformers (BERT) [1], a transformer-based model
illustrated in Figure 1, to analyze a plasma discharge
experiment. Our aim is to predict vertical plasma
instability and identify the underlying phenomena
responsible for these instabilities.
Since plasma with a D-shaped cross-section havedemonstrated high performance, future tokamak devices
will use elongated plasma [2]. However, more accurate
vertical control and predictions will be necessary, aselongated plasma are vertically unstable and can cause
disruptions in the plasma. To solve this problem, it isimportant to predict plasma vertical instability to control
it safely. In order to address this complex problem, we
utilized machine learning techniques, specifically neuralnetworks and deep learning algorithms, to develop

Figure 1. The structure of predictor of plasma verticalposition. Arrows represent flows of a input data. Given
an input length of m, and with h representing the head,
let X denote the input. The encoder comprises a selfattention layer and a feedforward neural network(FFNN). By leveraging BERT's token, segment, andposition embeddings, the model can generate dynamic
word representations, enabling exceptional performanceacross a wide range of natural language processing tasks.

predictive models that can predict plasma verticalinstability. The recent works have shown theeffectiveness of using machine learning approaches
based on experimental data for predicting plasma
disruption prediction [3] and controlling the plasma
shape [4]. The recurrent neural network (RNN) basedmodels such as Long Short-Term Memory have been
conventionally used to predict plasma disruptions using
time-series data.
Transformer-based models such as Generative Pretrained Transformer 3 (used for ChatGPT) have beenutilized for tasks like time series prediction, languagetranslation, and search engine functions. These models
have achieved higher performance compared toconventional RNN-based and convolutional neuralnetwork (CNN) based models. The transformer's
drawback of needing substantial amounts of time andtremendous data to train the model was a big problem.
However, this issue has been solved recently usingtransfer learning, which enables efficient training of themodel with small amounts of data by utilizing pretrained weights from a larger dataset, thereby reducing
the required computing resources.
The objective of this study is to identify the precursors
and correlations of the observed and controlledparameters, and to forecast plasma instabilities within thenear future, with a focus on predicting 0.01 seconds
ahead. To achieve this goal, we used measured data of
Kyushu University Experiment with Steady-state
Spherical Tokamak (QUEST) [5]at three different points
in time whose interval is 0.002 seconds. The
relationships between the parameters related to plasma
vertical displacement were analyzed, and we can predictthe plasma vertical instability. This result serves as thebasis for developing a predictive model that is capable offorecasting the plasma instabilities with high accuracy,
up to 0.01 seconds into the future. We can also interpretthe precursors and correlations of the parameters using
BERT.
References[1] Devlin, Jacob, et al. Bert: Pre-training of deep bidirectionaltransformers for language understanding, arXiv preprint
arXiv:1810.04805 (2018).
[2] Yushmanov, P. N, et al. Scalings for tokamak energy
confinement, Nuclear Fusion 30.10 (1990): 1999.
[3] Vega, et al. Dsruption prediction with artificial intelligence
techniques in tokamak plasmas, Nature Phys. 2022, 18, 741–750.
[4] Degrave, Jonas, et al. Magnetic control of tokamak plasmas
through deep reinforcement learning, Nature 602.7897 (2022):414-419.
[5] Hanada, Kazuaki, et al. Steady-state operation scenario andthe first experimental result on QUEST, Plasma and fusion
research 5 (2010): S1007-S1007.