Domain Generalization for Time Series: Enhancing Drilling Regression Models for Stick-Slip Index Prediction
Authors: Hana YAHIA
Bruno Figliuzzi
Florent Di Meglio
Gerbaud
Stephane Menand
Mohamed MAHJOUB
Venue: Accepted by TMLR ()

Abstract: This paper provides a comprehensive comparison of domain generalization techniques applied to time series data within a drilling context, focusing on the prediction of a continuous Stick-Slip Index (SSI), a critical metric for assessing torsional downhole vibrations at the drill bit. The study aims to develop a robust regression model that can generalize across domains by training on $60$~ second labeled sequences of $1$~Hz surface drilling data to predict the SSI. The model is tested in wells that are different from those used during training. To fine-tune the model architecture, a grid search approach is employed to optimize key hyperparameters. A comparative analysis of the Adversarial Domain Generalization (ADG), Invariant Risk Minimization (IRM) and baseline models is presented, along with an evaluation of the effectiveness of transfer learning (TL) in improving model performance. The ADG and IRM models achieve performance improvements of $10\%$ and $8\%$, respectively, over the baseline model. Most importantly, severe events are detected $60\%$ of the time, against $20\%$ for the baseline model. Overall, the results indicate that both ADG and IRM models surpass the baseline, with the ADG model exhibiting a slight advantage over the IRM model. Additionally, applying TL to a pre-trained model further improves performance. Our findings demonstrate the potential of domain generalization approaches in drilling applications, with ADG emerging as the most effective approach.

Published in Transactions on Machine Learning Research (03/2025)
Domain Generalization for Time Series: Enhancing Drilling Regression Models for Stick-Slip Index Prediction
Hana YAHIA
Centre Automatique et Systèmes (CAS)
Mines Paris, PSL University DrillScon, Helmerich & Payne
Bruno FIGLIUZZI
Center for Mathematical Morphology (CMM)
Mines Paris, PSL University
Florent DI MEGLIO
di _ htw..ddemmmiiiiimnnnsrrrrrserrr
Centre Automatique et Systèmes (CAS)
Mines Paris, PSL University
Laurent GERBAUD
lourent Dourrat.erbauddimineeyrrr.....pp
Cendre de Géosciences
Mines Paris, PSL Universily
Stephanc MENAND DrillScon, Helmerich & Poyne
Stephane. Stephane. www.eeeeeeaandddn.e.cc.oo
Mohamed MAHJOUB Drillscon, Helmerich & Payne
Mokamed, MokkameedMMaajjouu///pss.....c
Reviewed on OpenReview: https: // // openreview. net/ forum? id= PWWPPPJVL
Abstract
This paper provides a comprehensive comparison of domain generalization techniques ap- plied time series data within drilling context, focusing on the prediction of continuous Stick-Slip Index (SST), critical metric for assessing torsional downhole vibrations at the drill bit. The study aims to develop robust regression model that generalize across domains by training on 60 second labeled sequences of 1 Hz surface drilling dala to predict the SSI. The model is tested in wells that are different from those used during training. To fine-tune the model architecture, a grid search approach is employed to optimize key hy- perparameters. comparative analysis of the Adversarial Domain Generalization (ADG), Invariant. Risk Minimization (TRM) and baseline models is presented, along with an evalu- ation of the effectiveness of transfer learning (TL) in improving model performance. The ADG and IRM models achieve performance improvements of 10% and 8%, respectively, over the baseline model. Most importantly, severe events are detected 60% of the time, against 20% for the baseline model. Overall, the results indicate that both ADG and IRM models surpass the baseline, with the ADG model exhibiting a slight advantage over the IRM model. Additionally, applying TL to a pre-trained model further improves performance. Our find- ings demonstrate the potential of domain generalization approaches drilling applications, with ADG emerging the most effective approach.

---

Published in Transactions on Machine Learning Research (03/2025)
1 Introduction
Despite recent advancements, developing robust and generalizable machine learning (ML) model for pre- dicting drilling malfunctions remains a siguificant challenge. A primary reason lies in the quality of data, which must be both large and diverse to encompass the variety of cases needed for effective model training. However, real world datasets often face issues such as bias, incompleteness, or insufficient. labeling. Another significant challenge is dataset shift. Traditional machine learning models usually assume identical distribu- tions for training and larget data, an assumption that rarely holds in real world applications, complicating generalization further in fields like drilling.
During drilling, three types of vibrations can occur based on their direction: axial, lateral, and torsional. They can exist separately, which is rare, but usually they are synchronized in coupled modes. These vibra- tions can result from the bit-rock interaction or from the contact between the drill string and the borehole. In the present paper, we focus on torsional vibrations and more specifically on their most destructive type: the so-called stick-slip phenomenon. Stick-slip is among the most damaging types of vibrations that can affect the drill string, as reduces rate of penetration (ROP), slows down the drilling process, and decreases efficiency (Zhu et al. 2014). This phenomenon results in periodic, irregular downhole rotation speed, where the bit rotational speed alternates between sticking (complete stop) and slipping (surpassing the surface rotation speed multiple times) phases (Shen et al. 2017). Stick-slip occurs when the rotational energy in the drill string is insufficient to overcome the torque on the bit (TOB). While drilling, the top drive continues to supply rotational energy to the drillstring, but a sticking phase can occur in ils lower section, lasting bet ween to s due to the high TOB, The accumulated torsional energy is then abruptly released, resulting in a sudden spike in bit velocity during the slip phase. Throughout the stick-slip vibrations, the surface rotation speed can remains constant.
Advanced downhole sensors allow for precise detection and measurement of stick-slip vibrations. However, these sensors can be costly, and real-time transmission of downhole data to the surface is not feasible with standard methods like mud telemetry. While faster data transmission is possible using wired drill pipes, it comes with significantly higher costs. Additionally, mud telemetry's bandwidth limitations restrict surface transmission to low-frequency data, meaning high-frequency data can only be accessed after drilling is completed. This is why we aim to detect stick-slip vibrations using surface measurements.
With advancements in machine learning models and data analysis techniques, many researchers have adopted data-driven approaches to automatically detect downhole events, thereby reducing the need for frequent crew inspections (Saadeldin et al., 2023 Elahifar and Hosseini 2024). For instance, Zha and Pham (0118) devel- oped binary classification model utilizing surface data at 100 Hz to detect stick-slip, while Baumgartner and van Oort (2014) classified high-frequency downholc accelcration data at 400 Hz into stick-slip/ sttick-slpppnooo-stiicc no-stick- slip and whirl/ no-whirl categories. Additionally, Heggde et al. (2019) compared machine learning algorithms, including logistic regression, support vector machines (SVM), and random forests, to classify stick-slip sever- ity based on surface measurements. These last models were tailored for specific geological formations under the assumption that lithology, bottom hole assembly (BHA), drill bit, and drilling fluid remain constant. If any of these parameters change, new models must. be trained using updated data to reflect those changes.
A common challenge with data-driven machine learning models for detecting downhole vibrations is their limited ability to generalize effectively. These models often perform well on wells within the same drilling field as the training data but experience significant drops prediction accuracy when applied to wells outside this distribution, primarily due to differences in data distributions between training and test wells (Fang et al. 2020). Consequently, training new model for each test well often yields better results.
To address this issue in drilling applications, we explored various data normalization methods to enhance model generalization, as well as the application of Transfer Learning (TL) techniques to develop a more robust and adaptable model for predicting the Stick-Slip Index (SSI) (Yahia el al. 2024a). Additionally, some researchers have incorporated physics- based principles into model training to improve generalization. For example, a Physics-Informed Machine Learning (PIML) approach (Sheth et al., 2022 utilized historical well logs and prior stand data to predict upcoming stick-slip classes, integrating a physics- based Stick- Slip Index (SST) as a feature to boost classification accuracy. Similarly, incorporated physical features
2

---

Published in Transactions on Machine Learning Research (03/2025)
alongside surface measurements to develop a more generalizable model for predicting the SSI (ahia et al. 2024b).
Beyond the drilling application, numerous methods have been proposed to enhance the generalization of machine learning models. Recent advances foeus on aligning feature distributions between source and target domains through Domain Adaptation (DA) techniques, aiming to reduce domain discrepancies and improve target domain performance using existing source data. Some approaches achieve this by reweighting or selecting samples from the source domain (Borgwardt et al., 2006 Gong al. 2013), while others transform
the feature space to map the source distribution onto the target (Baktashmotlagh et al. 2013 Gopalan et al,, 2011). A key factor in these methods is how the similarity between distributions is mcasured. One approach matches distribution means in a reproducing kernel Hilbert space (Huang et al. 2006), while Fernando et al. [Fernando et al., 2013) propose mapping the principal axes of the distributions, and Ganin (Ganin and Lempitsky 2015) introduce Adversarial Domain Adaptation (ADA), which modifies feature representations rather than relying on reweighting or geometric transformation. This approach, which employs a deep and
discriminately trained classifier to measure distribution separability (Singhal et al. 2023 Li et al. 2024:
Fang et al. 2024), is extensively used across fields like task and text classification (Km et al. 2017 Xu
ct al.
2019
image classification for crack detection (Weng et al. (002)). Medical Image Analysis (Kollias
et al.
2024). sentiment analysis (Ganin and Lempitsky 2015 Shen et al. 2018 Li et al. 2017 Xia et al.
2023 or Named Entity Recognition (NER) (Naik and Rose 2020).
However, Domain Adaptation (DA) can only be applied when target domain data is available during training. In cases where this is not feasible, more challenging and realistic approach, known as Domain Generalization (DG) (Matsuura and Harada 2020), is preferred for practical applications. The primary objective of DG is to train a model using one or multiple different but related source domains so that it can generalize effectively to unseen target domains. While ADA was originally proposed for domain adaptation, the adoption of this reasoning has motivated adaptations of this approach for DG (Sicilia et al. 2023 Matsuura and Harada, Wang et al. 2022 Zhou et al. 2022). Domain gencralization has beon widely tested, particularly on image datasets for object recognition tasks (Albuquerque el al. 2019), Fault Diagnosis (Zhao and Shen,
2020
2022
Li et al. 2020), anti-spoofing (Liu et al. [022] Jia et al. 2020), depersonalized cross-subject vigilance
cstimation problem (Ma et al. 2019), and bearing fault identification (Zheng et al.) 2019).
While these (echniques have been extensively lested on image data, this paper focuses on testing and com- paring their effectiveness on time series data. Specifically, we compare a baseline model with an Adversarial Domain Generalization (ADG) and an Invariant Risk Minimization(IRM models to predict the severity of downhole torsional vibration (SSI) using 60 second sequences of 1 Hz surface drilling data as inputs within a real drilling context. The models are tested on separate test wells not used in training, and the ADG and IRM are applied to improve the baseline model predictions. Additionally, we investigate the benefits of Transfer Learning (TL) on trained models, and compare their performance post-TL.
The paper is organized as follows: Section 2 provides a brief overview of the rotary drilling system and stick- slip vibrations. Section 3 outlines the theoretical framework of the tested techniques. Scction e describes the models architectures, while Section 5 focuses on the training process and the results. Finally, concluding remarks are listed in Section
G
2 Description of Rotary Drilling System and Stick-slip Vibrations
2.1 Description of rotary drilling system
Rotary drilling is the most common method of drilling in oil & gas and geothermal applications. As depicted Fig. D it consists in using a drilling device called bit, which usually destroys the rock in one of the two following ways: either by a shcaring action (drag bits) or by indentation (roller-cones bits). The bit is linked to a rig located at the surface by a drill string comprising a series of pipes and a lower part called Bottom-Hoe Hole Assembly (BHA). The drill bit has to be rotated and pushed against the rock formation. The force, or Weight On Bit (WOB), required for the bit cutters to engage the formation is obtained from the weight of the drill string. The rotation is applied either by the use of rotary table or a top drive at the surface, and with assistance of downhole motor in some applications. Downhole motors are particularly
3

---

Published in Transactions on Machine Learning Research (03/2025)
used for directional wells and when the bit technology requires higher rotary speeds that are not technically possible with only rotation from the surface. They help avoiding drill string twist offs by allowing most of the rotational torque to be concentrated near the bit instead of lost through torque and drag on the drill string (Andrson et al., 1990). In addition, drilling fluids and a circulation system are used to suspend and carry the cuttings and to cool down the drill bit and downhole equipment. Other equipment can be used to improve the drilling efficiency such as shock subs damp the vibrations caused while drilling hard formation, or drilling jars that are used to free the drill string by delivering an impact load when stuck pipe is faced [Richard] 2001).
Top
BHA
Olgit
TOB
WOR
Contact forces
motor
drill bit
Figure Simplified schematic representation of the drilling system
2.2 Drilling torsional vibrations: Stick-slip
Stick-slip is a periodic low-frequency torsional oscillation characterized by fluctuations in the bit rotation speed, transitioning from zero during the stick phase to several times the surface rotation speed during the slip phase. In cases where a downhole motor is utilized, it generally maintains non-zero bit rotational velocity, however, severe stick-slip can still arise. To quantify the severity of stick-slip, we caleulate the stick-slip index (SST), industry-standard metric that normalizes the fluctuations in bit rotation speed (WBR) over specified time period, defined as follows:
SST = max Whit min wntt
(1)
WBit
Fig. 2 depicts the changes in surface torque and downhole bit rotation speed over time throughout a severe stick-slip sequence. The bit experiences two distinct phases: the stick phase, during which the bit rotation speed is zero, and the slip phase, where the bit rotation speed reaches twice that of the surface rotation speed. While the surface rotation speed remains constant imposed at the surface, the surface torque fluctuates in response the variations in bit rotation speed.
Our objective is to train generalizable regression model to detect stick-slip based on surface measurement. The model's 's inputs are 60 second sequences of surface features, including: surface torque, surface WOB, Rate Of Penetration (ROP), flow rate, and total rotation speed variations (surface rotation speed and downhole motor rotation speed), and the output the predicted SSI. To ensure the model's generalizability wells beyond those used in training, we explore the application of domain generalization techniques: ADG and TRM.
4

---

Published in Transactions on Machine Learning Research (03/2025)
Bit
Surface
Surface torque (lbf.ft)
18
Rotation speed (rpm)
178
12h
13k
00:50:20
02:50:30
B2:50:40
02:50:30
02:51:00
02:30:20
02:30-30
02:50-40
02:30:30
02/51:00
May
May 12,
Date Time
Date Time
(a) Surface lorque.
(b) Surface and bit rotation speed.
Figure 2: Example of sequence with severe stick-slip.
3 Methodology
Generalization in machine learning refers to the capability of model to perform effectively well on an unknown target domain that differs from the source domain on which it was originally trained. For drilling applications, we consider a training set comprising data from multiple wells, where each well is defined by distinct characteristics. Some are known, such as length, trajectory, and bit diameter, whereas others are unknown, uncertain or changing, such as rock characteristics or bit fatigue. The wells originate from different sites across various geographical locations and correspond therefore to distinct domains having their own unique data distribution. The goal to train a model capable to achieve minimal prediction error when applied test data from a new well, which constitutes different domain with a data distribution not encountered during training. Unlike traditional machine learning approaches, which assume that training and testing data originate from the same distribution, the concept of model generalization focuses on achieving robust performance across various domains without prior knowledge of the target domain.
During training, we typically have access to labeled data from one or several source domains. For the target domain, we can encounter various situations depending on the nature the available data:
Only unlabeled samples from the target domains: this scenario called unsupervised domain adap- tation.
Unlabeled samples from the target domains plus few labeled target samples: this scenario is referred to as semi-supervised domain adaptation.
No samples from the target domain during training, meaning that the specific target domain in which the model will ultimately be deployed is completely unknown. This situation is called Domain Generalization (DG) and is the one considered in the present article. We present the two tested approaches of DG, ADG and IRM, in subsections 3.1 and 3.2 respectively.
When a small number labeled target samples become available during testing, they can be utilized to further improve the performance of the trainod model on the test data. This process, known as Trans- fer Learning (TL) (Zhunng et al.) (2020), involves fine tuning the trained model to better fit the specific characteristics of the new dataset. (target domain).
5

---

Published in Transactions on Machine Learning Research (03/2025)
3.1 Adversarial Approach for Domain Generalization
Let us denote by X' the instance set of measurements conducted on the wells. From a mathematical per- spective, a domain is defined as a specific distribution D on the instance set X. In our context, each well constitutes its own domain on X. Suppose we have access to Ng wells with labeled data, characterized by the source domains { Our objective is to construct a regression algorithm using data from the source domains that generalizes well to new domains.
To specify a learning problem, two ingredients are required: a distribution D on the instance set X, and an unknown target function f X E that we seek to approximate. A classical approach for approximating f is to map instances into a feature space 2 using a representation function R: :: X' - 2 and then select a function h from a hypothesis class H that utilizes these features to perform the regression task. The representation function R induces a distribution on the feature space 2, which we denote by D. Furthermore, we denote by f: f + R the function satisfying f(x) = f o R(x) for all x E X.
In our case, we dispose of Ns labeled dataset {Di where D's = consists of m, measurements sampled iid from the source distribution D'z. Our objective to select a function h from the hypothesis class H. which minimizes the expected risk er on the target domain, defined as:
().,,, f( ))}
(T(h) = EEND, (1((s)), [(=)) = EE R(x), f(x)),
(2)
where L is the loss function used to predict the difference between the prediction h(2) and the target value [(2) and is usually selected to be the Mean-Squared Error (MSE), D denotes the original distribution of the target data, while Dr represents the distribution of the target data transformed by the representation function R. The main challenge here is that labeled data are not available for the target distribution, making the prediction of target error impossible to estimate directly. As a consequence, we need to rely on the available training data from the Ng source domains and to adapt the training procedure to ensure good generalization.
Theoretically, it can be shown that to minimize the expected risk er on a new domain using the available training source domains, one effective approach is to minimize the empirical risk on these source domains while simultaneously learning a representation that aligns the features across the source domains (Mansour et al. 2009; Albuqqueqqe et al. 2019 Ben-David et al.) 2006). Domain generalization algorithms aim to construct shared representation for all source domains, which still ensures strong regression performance. In this article, we rely upon the adversarial approach presented in [his paragraph| to jointly learn a shared features representation for the source domains and predict the stick-slip index. These approaches to domain generalization are theoretically justified under the following assumptions (David et al. 2010
Covariate shift: the target function f should remain consistent across all domains.
Similarity between features distributions accross domains: the representation R should result in similar features for all source domains. To quantify this similarity, the H-divergence is commonly used (Ganin et al. 2016 Ben- et al. 2006 2010 Kifer et al. 2004). In practice, the H- divergence (D's, D') between the source domains D's and D is estimated by training a classifier C' to distinguish between domains i and j based on the features produced by the representation R:
d (D's,, D2)
2erra(C),
(3)
where the classification error is estimated based on the empirical datasets D's and D with respective sizes m, and my
my +amy
(CT=(C)== mi + m,
Σ
La 1py
(4)
In Eq. (4) Let is classification loss, usually the cross-cntropy loss and 1 pg (2) is the indicative function taking the value when z comes from datasct D's. A high classification loss suggests that the classifier struggles to differentiate between the source domains. Conversely, a small H-divergence implies high similarity between the distributions D, and D,.

---

Published in Transactions on Machine Learning Research (03/2025)
Eristence of suitable mapping function: There must exist function h" that can accurately map features to their respective labels, regardless of the domain.
In our case, before applying DG to the regression model for SSI prediction using surface measurements, it is crucial to verify the validity of these assumptions. According to (David al. 2010), only the last two assumptions necessary for applying model generalization. Moreover, the covariate shift assumption is considered less restrictive, as it can be validated independently of the feasibility of model generalization. Regarding the second assumplion, there is always degree of similarity between feature distributions even when the imput feature distributions differ. This is because drilling operations typically rely on standardized tools and techniques, which help maintain consistency in surface mcasurement patterns across wells. Conse- quently, the overall structure of input feature distributions remains comparable across domains. For the last assumption, prior studies and data analysis have demonstrated correlation between surface measurements, particularly surface torque, and the severity of stick-slip, supporting the assumption that suitable mapping function exists. Furthermore, machine learning models have proven effective in capturing these relationships in similar applications, including detection of other downhole vibrations. For example, our work in Yahiia ct al. 2024a) demonstrates the fcasibility of developing a mapping function for SSI prediction. Even if the mapping function is not perfect, a regression model can still approximate it with a reasonable degree of accuracy.
Domain adversarial training is commonly used for lcarning domain-invariant features. This approach was originally introduced by Ganin and Lempitsky (Ganin and Lempitsky] [015 Ganin al.) 2016) in the context of domain adaptation. As described in section 3.1] minimizing the target error er amounts to reducing both the expected risks <s; on each source domain and the discrepancy between source domains. To that end, we aim to learn a representation R that aligns the source domains but still produces features that remain relevant. for the regression task al hand (Ganin and Lempitsky 2015). To achieve this, adversarial training is used to learn gencrator and a discriminator an adversarial manner. The input x is processod by an embedding function or representation R (generator) to learn domain invariant feature representation, which used input to the regression model h for mapping to the stick-slip index. To ensure alignment between the source domains, domain classifier C' (discriminator) is trained to distinguish between the source domains based their embedded features. The goal is to maximize the domains classifier's s loss, indicating that the embedded features from the source domains are sufficiently similar, making it difficult for the classifier to distinguish between them. For a dataset constituted of m observations {(x,, f(x)) ~ D's a the final loss function is:
well
invariatt features by simultaneously training two components: a
min 1 Σ E ((((((())))) A Lel (C(R(1))), 1 (m.)) 2m1
(5)
where the second term is used to penalize the distance between the embedded features for the source and target domains, La being the classification loss (cross-entropy loss) computed across the source domains, x being a weighting coefficient, and 1 D is the indicative function that takes the value the element z; belongs to the source domain D& and otherwise.
3.2 Invariant Risk Minimization (IRM)
Invariant Risk Minimization (IRM) is DG technique aimed at lcarning new data representation where the correlations remain invariant across different training environments, similar to ADG (Arjovsky el al., 2019). Specifically, the model learns a representation R(x), where ø represents the input features. The goal is for this representation to capture the underlying structure of the data invariant across the different environments. IRM seeks to identify a domain-invarian predictor g = p o R by searching for a regression Ne function p that is simultaneously optimal for all domains [D) which lead to the optimization problem:
7

---

Published in Transactions on Machine Learning Research (03/2025)
N Dgg (po R)
R. = arg min R.X -
(6)
subject to p E arg min EDy (po R), I,...,Ns,
p
where epy (p R) is the empirical risk associated with the i-th domain. Eq. 6 is challenging bi-level optimization problem. However, as demonstrated in (Arjovsky et al.) 2019), when a linear classifier p is fixed, any predictor 9 p R can be written 9 = P. for scalar P.- Without loss of generality, this scalar can be absorbed into R, such that R = P..R., resulting in = 1.R. Thus, R. itself becomes the invariant predictor. This reformulation simplifies the problem as follows:
R arg min RXX-R
Σ Dp (R) + ae V PIp==LOE (p.R)]1
where is a penalty weighting coefficient used to balance the ERM term and invariance of the predictor 1 R accross the different domains. We refer the reader to the original article (Arjovsky al. 2019 for detailed derivation of this result.
3.3 Transfer Learning
During the training phase, we do not have access to labeled target samples. However, during testing, some labeled samples may become available, which can be used improve the performance of the trained model in this spocific target domain. To achieve this, the trained model (source model) can be fine-tuned by adjusting all its parameters or selectively updating some of them using only the labeled target samples retraining data (Zhuang et al. (020). In our case, the source model is regression model trained in various source domains. The fine-tuning process, using the small set of labeled target samples, is fast as a result of the limited amount of data involved compared to what is required to train a new model.
4 Models Characteristics
4.1
Baseline model
To determine the baseline model, we observed that the existing literature on predicting downhole vibrations remains relatively limited, with no clear consensus on the optimal architecture for addressing this prob- lem. Recent studies have employed relatively straightforward machine learning techniques: for instance, (Saadeldin et al.] 2023) utilized methods such support vector machines (SVM), radial basis functions, and functional networks, while (Elahifar and Hosseini 2024) relied on decision trees for stick-slip predict ion. To the best of our knowledge, our approach (Yahia et al. 2024a) is among the first to leverage deep learning techniques for stick-slip index (SSI) prediction. We explored with various architectures, including Recurrent Neural Networks (RNNs), transformers, and Long Short- Memory networks (LSTMs). Ultimately, we selected the LSTM architecture, as demonstrated superior performance terms of both accuracy and execution time.
The baseline model architecture, based on Yahia al. (024a)) consists of single component that processes 60 second sequences of surface measurements as input and predicts the SSI. The architecture is built using a series of LSTM layers combined with Layer Normalization (LN) layers followed by an output dense layer with a single neuron to generate the SST prediction. The loss function used is the MSF.
Transformer model: In recent years, the transformer model (Vaswani 2017] has gained popularity over LSTM, as it eliminates the sequential processing of LSTMs by using self-attention mechanisms to process all time steps in parallel. This parallclization is often seen as advantagoous for time series tasks. However, in choosing the architecture for the generator, we compared LSTMs architecture with transformers. During testing, the transformer model required significantly more time to train (training time increased by a factor
8

---

Published in Transactions on Machine Learning Research (03/2025)
of five), and its performance was less favorable compared to LSTMs, which are more efficient with smaller datasets. For these reasons, opted to use LSTM layers for the generator architecture.
4.2 Adversarial Domain Generalization model
For ADG, we adapt the global model architecture proposed by Ganin al. (0166), which comprises three main components (sce Fig.
Generator (or feature extractor): This component processes 60 second sequences of surface measurements, generating D-dimensional embedded feature vector E RP. The generator param- eters are indicated by Ic. and the relationship is expressed 2 G(æ; fG), where G is the same as R. in Eq. (2). The generator architecture comprises a series of LSTT layers combined with LN layers, mirroring the baseline model except for the final output dense layer.
Discriminator (or domain classifier): The discriminator takes the embedded features vector z as input and classifies it based on annotated data from the source domain. Its parameters are represented by fc, and its output the predicted domain for each sequence: C(2;0c) (see Eq. ( The discriminator structured as fully connected neural network, with a Gradient Reversal Layer (GRL) as input layer reverse the direction of the gradients during backpropagation (multiply the classification loss by -À (see Eq.
This component utilizes the embedded features from the generator to predict the stick-slip severity index (SSI) for each sequence. The parameters for this mapping are denoted by 8ss7 and the regression output (ssr) (see Eq. 2 The SST-predictor is also fully connected neural network.
SSI-predictor:
Dense
Dense
Predicted
Imput
1
LSTM
LN
1
LSTM
LN
1
LSTM
LN
SSI predictor h(2; 85si)
Gradient reversal layer (GRL)
Dense
Generator 6(x;8))
Predicted domain label
Domain classifier fier C(2;0))
Figure 3: Architecture of ADG model. The generator is composed of a sequence of LSTM and LN layers; the SSI predictor and domain classifier are fully connected neural networks; Gradient Reversal Layer (GRL) is used to reverse the direction of the gradients during backpropagation.
During the training phase, our objective to minimize the SSI prediction loss for the training wells while op- timizing the generator parameters to make the embedded features z (where z = G(x; fla) domain-invante which involves aligning the source distributions (G(z,, fG), 2; ~ D.,} for E 21,2, 2.., N₂), where N, is the number of training wells. To achieve this, we aim find the generator parameters &g that maximize the domain classifier loss, promoting
and making the feature distributions across domains

---

Published in Transactions on Machine Learning Research (03/2025)
similar possible. Simultancously, optimize the domain classifier and the SSI-predictor parameters, Ac: and Ossr respectively, to minimize their losses. The corresponding optimization problem is given by
cc. ASsi &g = a a ar E(0g, fc, 8ssi) = L(fG, (ssi)
(8)
In Eq. L is the loss function used for the SSI prediction (MSE), while La the classification loss (cross-entropy loss), computed across the N, source domains. The parameters Oc of the domain classifier are optimized to minimize the domain classification loss (Eq. (9)), the parameters 83sı of the SSI-predictor are optimized to minimize the SST prediction loss, while the feature mapping parameters 0g are adjusted to
). The parameter À is a weighting coefficient. used to control the trade-off between the losses Ganin and Lempitsky 2015
both minimize the SSI prediction loss and maximize the domain classification loss (Eq. (10)).
Oc = =argagaaxe B(da, Oc, OSSI)
(9)
(0c, 0ssi) argmitloa. E(Og, 0c 0ssi)
(10)
4.3 Invariant Risk Minimization model
The IRM model adopts the same architecture as the ADG model, with the primary difference being the exclusion of the domain classifier component. Specifically, the IRM consists of the generator G, made up of series of LSTM and LN layers, which processes 60 second sequences of surface measurements as input. The output from the generator is then fed into the SST predictor h (same ADG), fully connected neural network, which is responsible for predicting the Stick-Slip Index (SSI) (see Fig. 3 An additional trainable variable 3 is introduced to represent the linear classifier in Eq. 11 and to compute the regularization term. So that the IRM loss function is:
Ns Σ L(h(G(ED) (o) (ss.) al/V, p-10 fc),0ssi 0s)) f(rpg) (11)
1)1
Where is the MSE loss, TD: the data from the i-th source domain, (( represent the true SSI of DD. and is a penalty weighfing coefficient.
5
Experiments
In this section, present a variety of results for training machine learning model to predict the SSI using 60 second sequences of surface measurements input. The model is trained using different training wells (source domains) and tested on completely different wells (target domain) in four different approaches:
Baseline model: A traditional deep learning model consisting of a single component that takes the 60 second sequences of surface measurements as input and predicts the SSI as output.
ADG model: As explained subsection 4.2 the model consists of three components: a generator, SS1-predictor and domain classifier.
IRM model: This generalization technique incorporates regularization term into the loss function. It processes 60 second sequences of surface measurements as input for the generator, and the output is then used as input to the SSI predictor. The architecture is illustrated in subsection 4.3
TL: In contrast the previous techniques, TL is applied during the testing phase. Once model trained (either the ADG, IRM, or baseline model), labeled surface data sequences from the test well are available during testing, we apply TT. technique to improve the model's performance.
10

---

Published in Transactions on Machine Learning Research (03/2025)
5.1 Data processing
Extensive evaluation are performed of the proposed approaches in Table I Some of these wells are from the same field and share similar characteristics. To assess the generalizabiiity of the trained model, we ensure that wells from the same field are all consistently used either as test data or training data, but not both. The first three wells, which originate from the same field and represent a substantial number of sequences, were selected for the training process. The last three wells were reserved for testing, as two of them are from the same field, with an additional distinct well included to further evaluate the model's generalizability. The remaining wells (4, 5, and 6) were used alternatively for training and validation (see section 5.2
number of distinct wells outlined
Table 1: Source and larget well characteristics, where vertical wells are drilled straight down from the surface into the target formation or reservoir, and lateral wells are drilled vertically to certain depth, then deviates horizontally within the target.
Well length (ft)
Bit diameter (inch)
Drill Pipe
BHA
Number of sequences
Field number
Type of wellbore trajectory
Well
diameter (inch)
length (ft)
Lateral
18 120
8 3/4
5
167.94
33 303
2
Lateral
18 204
8 1/2
5
166.9
50 285
3
Lateral
17 921
8 3/4
5
136.94
15 147
4
lateral
27 723
8 3/4
5 1/2
101.03
52 008
Lateral
19 355
6 6/4
5 1/2
128.99
119 613
6
Lateral
29 077
8 3/4
5
130.52
255 687
7
Lateral 28 645
8 3/4
5 1/2
97.36
449 002
8
Vertical 20 619
8 3/4
5
110.31
22 054
9
Vertical 20 679
8 3/4
5
111.23
136 824
For each well described in Table [ 1 Hz time series of surface and downhole drilling data are collected. The surface data, which includes surface torque, surface weight on bit, rate of penctration, flow rate, and total rotation speed variations, divided into 60 second sequences where each sequence serves as an input sample for the regression model. The downhole data contains the bit rotation speed used to label the surface data. For each 60 secondsequence, we inspect the downhole bit rotation speed and we calculate the true SSI described in Eq.
5.2 Grid search for hyperparameter tuning
To tune our model's hyperparameters, we employ the traditional grid search method with the ADG model. This approach involves exhaustively searching through specified subset of the hyperparameter space for the training algorithm. The hyperparameterss evaluated include the regularization parameter for the generator (covering bias, kernel, and recurrent regularization), the number of hidden layers in the generator, and the loss function weighting coefficient À for the ADG (Eq. 5)- The potential values for the regularization coefficient are (10-3, 3 10 10-5), the number of hidden layers in the generator are set to (4, 6, 8), while the weighting coefficient varies among (1, 10, 100, 1000). In our experiments, validation data is used determine the optimal hyperparameter values. Since our goal is develop a model that generalizes effectively accross sources, select validation data from wells entirely distinct from those used in training. Therefore, in each training session, from the first six wells listed in Table [ four wells are used for training and the remaining two for validation. The last three wells (from two different rigs) are reserved for testing after the hyperparameters have been selected. Selecting the two wells for validation among the first. six wells is challenging, as this choice significantly influences both model performance and hyperparameters selection: when wells with characteristics similar to the majority of training wells were chosen, the SST validation error tended to be low. In contrast, choosing wells with different characteristics resulted a higher validation error. To address this, we test. three different validation data cases, outlined in Table 2 In each case, the
11

---

Published in Transactions on Machine Learning Research (03/2025)
first three wells in Table I which are from the same field, are fixed training data, and wells 4, 5, and 6, from 3 different rigs, are used alternately to select two wells for validation in each scenario.
Table 2: Validation data selection for the three lested cases.
Well 4
Well 5
Well 6
Case
Validation data
Training data
Validation data
Case
Validation data
Validation data
Training data
Case 3
Training data
Validation data
Validation data
5.2.1 Regularization coefficient
Due to limited computational capacity, we do not search for all the hyperparameters simultancously. Instead, we begin by fixing the number of hidden layers in the generator at 6 while varying the regularization coefficient and the weighting coefficient for the ADG across the different validation data cases described in Table 2 The architectures for the domain classifier and SSI predictor were fixed at 5 dense layers, each containing [60, 40, 20, 10] neurons for the first four layers, and 6 neurons (representing the number of training and validation wells) and 1 neuron (for SST) for the last layers, respectively. The generator architecture comprised LSTM layers combined with 5 layer normalization (LN) layers (including 2 input layers, hidden layers, and output layers), with 64 neurons per layer and bias, kernel, and recurrent regularization applied uniformly across all layers. Each architecture search was conducted for 500 epochs, utilizing a constant learning rate of 10-3 an Nvidia GPU. For cach combination of hyperparamoters, and for each validation data case, the model was trained using three different initializations For each initialization, the trained model was used to calculate the MSE and dynamic time warping (DTW) [Li 2021) which is computational technique used to measure the similarity between two time series that may vary in speed or timing, between the actual and predicted SSI validation data. The average of the three initializations was then computed. The key advantages of DTW are ils robustness to outliers, making less sensitive to abnormal points, and ability stretch or compress parts of the time series to achieve optimal alignment. In contrast, MSE calculates poini-by-point differences in magnitude between the two time series, without accommodating variations in timing alignment. However, MSE is generally easier to interpret, as it measures the average of the squared differences between predicted and actual values, making it more straightforward than DTW. To facilitate comparisons, we decide to normalize the DTW by the number of corresponding well sequences.
Fig. | illustrates the average MSE on validation data across the three tested initializations for each of the three cases. As we can see, results vary across cases, with case exhibiting the lowest MSE SST validation error. This outcome is likely due to the training dataset's sequence configuration, as for case 2, wells 4 and 5 were selected for validation, while well 6, with the highest sequence number, was included in the training dataset. Fig 5 displays the mean MSE and normalized DTW on validation data, caleulated over the three tested cases. Notably, the crror variations remain consistent whether using MSE or DTW. The results indicate that the lowest validation SSI prediction error is altained with a regularization coefficient of 10-4 for most of the tested weighting coefficients. Therefore, this hyperparameter will be set at 10-4 for the remainder of the paper.
5.2.2 Number of generator hidden layers and weighting coefficient
To select the optimal number of hidden layers for the generator and the weighting coefficient À (see Eq. conduct grid search using three different initializations across the three validation cases (see Table following the same approach as for the regularization coefficient. The tested values for the generator's hidden
8
2
layers are (4, 6, 8) and for the weighting coefficient À we set (1, 10, 100, 1000). The optimal parameters are chosen based on the SSI validation error. Similar to the regularization coefficient selection, Fig D shows the average MSE on validation data across the three initializations for each case, while Fig 7 presents the mean MSE and normalized DTW on validation data, averaged across the three cases. We choose the hyperparameter configuration with the lowest SST validation error, which includes 6 hidden layers for the
12

---

Published in Transactions on Machine Learning Research (03/2025)
0.25
0.25
0.146=0.640
0.113 +0.007
0.139:0.012
0.035
O 0.034
0.039
0.20
0.20
Weightng confiicient
Weighting corfficient
0.139
0.126 60.023
0.128 12.0...000
0.127 a2.7..0.
0.039 60.016
0.038 0.023
100
0.135
0.137
0.10
100
0.106
0.032
0.078
00.0
-0.05
00.05
1000
0.125
0.121
0..09
1000
00.42:0.074
0.066
0.129
10-1
10
10
10
10
10
Regularization coefficient
Regularization coefficient
(a) Case
(b) Case 2
Weighting coeffrcient
-0.15
0.123
-0.0
0.05
1000
0.163:2.037
0.112
10
Regularizationn coefficient
(c) Case 3
Figure 4: Average MSE of SSI validation data predictions for the three tested cases (Table. 2 with varying regularization and weighting coefficient over three different intializations: The ADG model is trained for each combination of hyperparameters using three distinct initializations. The validation SSI error calculated for each one, and the average error is then computed across the three initializations.
generalor and a weighting coefficient of 10. These hyperparameters will be fixed for the remainder of the paper.
For the penalty weighting coefficient of the IRM model, we tested five different values (000100.111111100000 and selected the model with the lowest SSI prediction loss, which occurred with a penalty coefficient of This IRM model is then used for comparison with the ADG and baseline models for the rest of the paper.
Remark:
5.3 Results and discussions
5.3.1
Comparison of ADG, IRM, and Baseline Models
With the selected hyperparameters, we proceed to compare the ADG and IRM techniques applied to time series for SST prediction against the bascline model (scction[4.1) For training, the first six wells are designated as training data and the last three test data (Table with 10% of the training data reserved for validation. Each model is trained for 1000 epochs, with batch size of 2048, and fixed learning rate of 10-3 and
13

---

Published in Transactions on Machine Learning Research (03/2025)
0.25
0.25
0.091
0.075 #0.014
0.103
0.089 013
0.082 =0.010
0.097
0.20
0.20
Weightin confiicient
Weighting corfficient
0.120
0.096 80701
0.089 0..026
0.106
0.091
0.087 00.016
100
0.116
0.095
0.113 10.046
0.10
100
0.104
0.095
0.099n0.028
-0.10
-0.05
-0.05
1000
0.124:0.c36
0.117
0.120:0.am
1000
0.104
0.092 10.022
0.094s8.026
10-1
10 Regularization coefficient
10
10
10" Regularization coefficient
10
MSE
(b) DTW
Figure 5: Average MSE and DTW of SSI validation data predictions for the three tested cases with varying regularization and weighting coefficient in the ADG model.
we retain the model that achieves the lowest SSI validation MSE. We repeat the training five times, each time with a different initialization, and calculate the average test SST prediction error across the five runs. The ADG model requires about six hours of training, whereas the IRM and baseline models each take approximately four hours.
Table. 3 shows the normalized DTW results for the three test wells, over the five runs, for the ADG, TRM and baseline models. The ADG and IRM models demonstrate better generalization compared to the baseline, achieving improvements 10.86 % and 8.42 %. respectively. However, the ADG shows slightly better results than the IRM. Fig. g and Fig. IO illustrate examples of true and predicted SSI sequences for the three tested models across the three test. wells. As shown in Fig. 9a and Fig. 9h all models generally perform well in predicting SSI for most sequences. However, there are also sequences where they all fail to capture SSI variations, such as in Fig. 99] where they overestimate the SSI, and Fig, [d] where they underestimate the SSI. Notably, in some cases (Fig, 10 the ADG and IRM provide more accurate SSI predictions than the bascline. This indicates that the generator effectively projects the training data into a domain-invarian feature space, whether with ADG or IRM, enabling the SST predictor to make more reliable predictions using the newly projected features. Fig. [Ob and Fig, [0c show sequences where the ADG model predicts the SSI with slightly greater accuracy than the IRM model, with both models consistently outperforming the baseline.
To further evaluate the results, Fig. S presents the confusion matrices for the three tested models, averaged over the three test wells. The SSI is classified into four distinet categories based on the severity of stick-slip: sequences without stick-slip (SSI e [0, 0.3[) are assigned to the first class; sequences with low stick-slip (SSI E [0.3, 0.5l) fall into the second class; soquences with moderate stick-slip (SST [0.5, 0.7[) are grouped into the third class; and sequences with severe stick-slip (SSI > 0.7) are classified into the fourth class. As shown, the confusion matrices reveal that all models correctly identify the majority of sequences for each class, except for the last class, which corresponds to sequences with severe stick-slip, the most damaging sconario. The bascline model incorrectly assigns most of these sequences to the third class instcad of the fourth, a misclassification not seen in the ADG and IRM models. The IRM and ADG models show similar performance, with the IRM model predicting SSI more accurately for sequences with low SSI, while the ADG model performs better for sequences with high SST. The reasons for sequence misprediction are discussod in the subsection 5.3.3 In conclusion, the IRM and ADG models exhibit similar performance, with slight advantage for the ADG, but both outperform the baseline model.
14

---

Published in Transactions on Machine Learning Research (03/2025)
0.25
0.25
0.128=0.619
0.125 20.026
0.127
0.074
0.076
0.20
0.20
Weightng confiicient
Weighting corfficieet
0.144=0.com
0.121 80.015
0.128
0.055
0.035
0.040
100
0.130
0.126aam24
0.125
0.10
100
0.052 10. n..4
0.049 mo.om2
0.077
-0.10
0.05
0..5
1000
0.129:b:011
0.132
1000
0.190
0.058
0.192:6.229
Generator hidden layers
Generator hidden layers
(a) Case
(b) Case 2
0,110 s0.041
0.104 0.104m0.025
0.094
0.20
Weighting coefiiiint
10
0.087 =0.0000 828
0.085
0.078 =0.007
-0.55
100
0.156 10.00M
0,234
0.111 10.55.0.04
0..0
1000
0.135 20.000 D83
0,231=0.195
0.105 20.003
Generator Nidden layers
(c) Case
Figure 6: Average MSE of SST validation data for the three tested cases with varying generator hidden layer number and weighting coefficient over three different intializations: The ADG model trained for each combination of hyperparamcters using three distinct initializations. The validation SSI error is calculated for each one, and the average error is then compuled across the three initializations.
Table 3: Normalized SSI DTW of the trained ADG, IRMand bascline models on the three test wells: Each model trained with five different initializations, and the error is reported as the average across these five runs.
ADG Baseline (%)
IRM vs
ADG vs
Baseline
ADG
IRM
Baseline (%)
IRM (%)
Test well 1
0.136
0.122
10.29
0.120
11.76
-1.66
Test well 2
0.086
0.075
12.79
0.080
6.97
6.25
Test well 3
0.107
0.097
9.34
0.100
6.54
3.00
5.3.2 Transfer learning application
To enhance model performance on specific target well, a TL technique can be applied if a small amount of labeled surface drilling data for that well is available (see section. 3.3). our case, we test the contribution of TL each of the three test wells by using the first 10 % of labeled surface drilling data with both, the
15

---

Published in Transactions on Machine Learning Research (03/2025)
0.25
0.25
0.104
0.086
0.099 03.
0.095
0.089 80.015
0.095
0.20
0.20
Weightig confiicient
Weighting corfficient
0.095
0.080 60.018
0.08220.a1n
0.095 in
0.084 n0.044
0.086
100
0.112+0.057
0.137 29.058
0.106
0.10
100
0.100
0.098
0.091
00.0
0.05
0..5
1000
0.151
0.137
0.143:0.003
1000
0.108
0.099 a..
0.108
Generator hicden layers
Generator hidden layers
(a) MSE
(b) DTW
Figure 7: Average of the MSE and DTW of SSI validation data for the three tested cases with varying generator hidden layer number and weighting coefficient for the ADG model.
ADG and baseline models. The source model, either the trained ADG or baseline model, is then fine-tuned to adapt to the larget well by retraining only the weights and biases of the first two layers in the generator and SSI predictor for the ADG model, and the first two layers in the baseline model. The same optimizer and learning rate as the source model training are used, but with a significantly reduced number of epochs, resulting in lighter computational load, with a retraining time of approximately 1.5 minutes.
Table, presents comparison of model performance before and after applying TL for both ADG and baseline models. For each test well, the source model (either ADG or bascline) is partially retrained using the first 10 % of labeled surface drilling data. The results in the table show that TL enhances model performance in both ADG and baseline models, with a decrease in normalized DTW following retraining. Additionally, this TL approach is efficient in terms of time and requires only small amount of data compared to what would be needed train new model.
Table 4: Comparison of model performance, for both ADG and baseline models, before and after applying TL, using normalized SSI DTW as the evaluation metric across the three test wells: Each source model undergoos partial retraining with the first 10 % of labeled surface drilling data from each test well.
ADG
Baseline
Without TL
With TL
%
Without TL
With TL
%
Test well 1
0.122
0.095
22.13
0.136
0.110
19.11
Test well 2
0.075
0.058
22.66
0.086
0.06
30.23
Test well 3
0.097
0.088
9.27
0.107
0.100
6.54
Fig. 11 shows the actual and the predicted values of SSI over time for the three test wells, comparing results with and without the application of TL. As SSI better than the source one for both models. Additionally, even after TL application, the ADG model outperforms the baseline model, with a lower normalized DTW across all three test wells (see Table. 4
see, after TL was applied, the new model predicts the
5.3.3 Causes of misprediction of SSI
The three models, bascline, ADG and IRM, before or after the application of TL, mispredict the SSI for certain sequences. In this section, we analyze some of the potential reasons for these mispredictions, which cannot generally be addressed by domain generalization approaches.
16

---

Published in Transactions on Machine Learning Research (03/2025)
100
100
Cabes
53.77%
41.28%
4.53%
0.43%
Class 1
57.66%
38.14%
3.73%
0.445
-80
Cless 2
Cass 2
14.22%
44.47%
17.30%
24.02%
13.72%
47.04%
18.68%
20.59%%
Class Tue Class
Clss Thee Class
6.75%
29.29%
32.82%
31.13%
40
10.85%
19.27%
37.40%
32.48%
40
Class-
1.85%
33.64%
43.56%
20.95%
Class 4
11.24%
20.788
20.36%
47.62%
Predicted Class
Predicted Class
(a) Baseline
(b) IRM
80
22.11%
60
25.31%
41.69%
27.35%
20
2.30%
24.89%
16.20%
56.51%
Class 1
Class 2
Class 3
Class 4
Prodicted
(c) ADG
Figure 8: Confusion matrices illustrating the performance evaluation the baseline, IRM and ADG models over the three lest. wells. The true and predicted SST values are classified into four calegories: sequences with SSI E [0, 0.3[ are assigned to the first class; SST [0.3, 0.5[ are classified into the second class; if the SSI E [0.5, 0.7[ belong to the third class; and sequences with SSI 2 0.7 assigned to the fourth class.
Synchronization problem In drilling operations, time delay, commonly referred to as jet lag, occurs between surface and downhole measurements. This delay corresponds to the propagation time required for downhole data to reach the surface, and varies over time. Consequently, achieving perfect synchronization between surface and downhole measurements especially challenging, particularly during long-duration drilling.
To evaluate our trained models, predict the SSI using surface measurements and compare these predictions with the true SSI, which is calculated using synchronized downhole measurements (Eq. 1. However, due to imperfections in synchronization, discrepancies can occur, leading to errors in evaluation, as illustrated in the following example. The synchronization issue is present. in most domains and cannot. therefore be addressed with domain generalization techniques.
Fig. 12 shows long test sequence that exhibits synchronization issues between surface and downhole data, with the surface data being significantly shifted. In this sequence, used the 60 second red and blue
17

---

Published in Transactions on Machine Learning Research (03/2025)
True
Pred: Baseline
True
Pred: Baseline
Pred:
Pred: IRM
Pred: ADG
Pred: IRM
SSI
SSI
0.6
0.5
014
0.2
17:00
19.55
12:30
$3:35
17 40
1745
00:00 60
50046
01:50
00:51
00.00
00.53
Jun 7.
Date Time
Date Time
(a) Sequence
(b) Sequence 2
True
Pred: Baseline
True
Pred: Baseline
Pred: ADG
Pred: IRM
Pred: ADG
Pred: IRM
1.2
SSI
SSI
AM
0.2
13:30
13:40
13:50
14:00
D4110
34.00
10.00
13:30
19:35
10:40
301.0 2002
Dec 16, 2014
Date Time
Date Time
(c) Sequence 3
(d) Sequence 4
Figure 9: True and predicted SSI values over time for the three trained models: baseline, IRM, and ADG, based on sequences from the three test wells. In the first two sequences, all models accurately predict the SSI. In the last two sequences, all models fail to capture the variations SST.
soquences to predict their SSI using the trained ADG model. The model misprodicts the SSI of the red sequence, where the predicted SSI is high (0.77), due to surface torque variations, while the true SSI, calculated from the mismatched downhole measurements, is much lower (0.1) due to the absence of downhole vibrations. For the blue sequence, the predicted SSI is 0.84 and the true SSI is 0.92, indicating that the model predicts the SSI fairly well. This because surface torque variations are observed, likoly caused by downhole vibrations, even though they do not correspond to the true sequence.
Unobservable stick slip In some instances, downhole stick-slip can occur without generating vibrations detectable the surface. This suggests that the downhole vibrations were attenuated as they propagated to the surface, due to friction and viscous damping, making them undetectable with low-frequency (1 Hz) surface measurements. This phenomenon is more commonly observed in lateral wells, particularly within the horizontal section. Fig. 13 illustrates the variation in surface torque and bit rotation speed during a test sequence in the horizontal section of a Test well. As shown, severe stick-slip occurs at the bit (with a true SSI of 0.87), but no surface torque variation observed. Consequently, the trained model fails to predict the SSI using Hz surface data for this type of sequence (with a predicted SST of 0.14).
Mislabeled data To label surface data sequences, we compute the true SSI using downhole bit rotation speed measurements [Fq However, some sequences may exhibit, sudden variations (peaks) that do not
18

---

Published in Transactions on Machine Learning Research (03/2025)
True
Pred: Baseline
True
Pred: Baseline
Pred: ADG
Pred: IRM
Pred: ADG
Pred: IRM
SSI
SS
0.0
0.0
0.2
0.1
07:30
08:00
09:00
09:30
10:00
10:30
19:26
19:28
11,
Date Time
Date Time
(a) Test well
(b) Test well
True
Pred: Baseline
Pred: ADG
Pred: IRM
SSI
13:35
11:45
Dec 7,2004
Date Time
(c) Test well 3
True and predicted SSI values over time for the three trained models: baseline, IRM, and ADG, based on sequences from the three test wells. For the three test sequences, the ADG and IRM provide more accurate SSI predictions than the baseline. The last two sequences the ADG model predicts the SSI with slightly greater accuracy than the IRM model.
Figure 10:
signal stick-slip occurrences, but still result in high SSI values. Fig. 14 illustrates test sequence where sudden variation in downhole bit rotation speed results in high true SSI (0.72). However, corresponding changes are observed at the surface, causing the predicted SSI to be significantly low (0.21). This issuc arises not from the trained model, but from the sequence labeling process.
Domain mismatch Another potential cause of ADG or IRM issues is domain mismatch, which arises when the model not trained on enough data that adequately represents the largel domain. Consequently, the model has difficulty generalizing effectively. To enhance generalization, the model may need be trained a more diverse dataset, which would improve its performance when exposed to new, unseen data.
19

---

Published in Transactions on Machine Learning Research (03/2025)
True
Predicted before TL
True
Predicted before TL
Predicted after TL
Predicted after TL
SSI
SSI
0.8
0.8
May 12, 2023
09:30
4000
tn 0000
13:36
12:15
12:30
13:00
May 11, 2022
19:30
10:00
10,8.00
11:00
12:35
12:30
13000
Date Time
Date Time
(a) Test well 1: ADG model
(b) Test Baseline model
True Predicted after
Predicted before TL
True Predicted after
Predicted before TL
SSI
SSI
0.0
Du
M Morcn
D.0
1410
14:20
1643
24.30
24.00 Dec 2014
14:10
1420
1844
26/00
Dec 2014
Date Time
Date Time
(c) Test well 2: ADG model
(d) Test well 2: Baseline model
True Predicted after TL
Predicted before TL
True Predicted after TL
Predicted before
1.2
1.2
SSI
SSI
15.00 Dec 2014
18:30
15.00 Dec 21. 2014
18:00
18:30
Date Time
Date Time
(e) Test well 3: ADG model
(f) Test well 3: Baseline model
Figure 11: True and predicted SSI values over time, for both ADG and baseline models applied on sequences from the three Lest. wells before and after the application of TI.
20

---

Published in Transactions on Machine Learning Research (03/2025)
Max, mean, min bit rotation speed Surface torque (rpm)
15k
(lbf.ft)
10k
5k
06:55:00
06:55:30
06:56:00
06:56:30
06:57:00
400
300
200
100
06:55:00
06:55:30
06:56:00
06:56:30
06:57:00
Dec 2014
Date Time
Figure 12: Variation in surface torque (surface measurement) and bit rotation speed (downhole measure- ments) for test sequence. This sequence exhibits synchronization issue between surface and downhole mcasurements. The predicted SSI for the 60 second red and blue sequences are high (0.77 and 0.84 respec- tively) due to torque variations. However, when using the mismatched sequence, the true SSI values for the red and blue sequences are 0.1 and 0.92 respectively. Consequently, the model mispredicts the SST of the red sequence due to the synchronization problem.
Max, mean, min bit rotation speed Surface torque (rpm) (lbf.ft)
12k
11k
10k
12:41:00
12:41:30
12:42:00
12:42:30
12:43:00
23, 2023
200
100
12:41:00 Apr 23, 2023
12:41:30
12:42:00
12:42:30
12:43:00
Date Time
Figure 13: Variation in surface torque (measured at the surface) and bit rotation speed (measured downhole) during test sequence. Stick-slip behavior occurs at the bit, but surface torque variation is observed during propagation (due to friction). The predicted SS1 using the trained ADG model for the 60 second red sequence is 0.14 and the true SSI is 0.87.
21

---

Published in Transactions on Machine Learning Research (03/2025)
Max, mean, min bit rotation speed Surface torque (rpm)
14k
(lbf.ft)
12k
10k
8k
00:58:00
00:58:30
00:59:00
00:59:30
2019
100
00:58:00 Aug 29, 2019
00:58:30
00:59:00
00:59:30
Date Time
Figure 14: Variation in surface torque (measured at the surface) and bit rotation speed (measured downhole) during a test sequence. Tn this sequence, peak in downhole bit rotation speed during the 60 second red sequence generates a high true SSI (0.72), despite significant variation in surface measurements, resulting in low predicted SSI (0.21).
22

---

Published in Transactions on Machine Learning Research (03/2025)
6 Conclusion
In this paper, we explored the application of domain generalization techniques to time series data for pre- dicting the Stick-Slip Index (SSI) in drilling operations. Our approach used 60 second sequences of 1 Hz surface drilling data as inputs. We compared the performance of the Adversarial Domain Generalization (ADG) model with the Invariant Risk Minimization (IRM) model, traditional baseline model, and TL approach.
To optimize key parameters including the regularization coefficient, the architecture of the model, and the weighting coefficient for the ADG and the IRM models, we employed grid scarch methodology. Our results demonstrate that ADG and IRM models significantly enhance the generalization capabilities of the SSI prediction model, achieving improvements the ADG model slightly outperforms the TRM model. Additionally, the implementation of TL on a pre- trained model (whether ADG or baseline) demonstrated improved performance. Even after TL application, the ADG model outperforms the baseline model, which highlights the generator's ability to map the training data sequences into space where the SSI predictor cannot differentiate among them.
10% and 8% over the baseline model, respectively. Notably,
The proposed model can readily be used to detect stick-slip on new wells with no need for downhole sensors to be installed. In particular, severe events are detected 60% of the time. This constitutes a new tool for drilling operators to monitor drilling vibrations in real-time, and reduce drilling costs. In future works, we intend implement ADG using a larger variety of training wells. Additionally, we plan to explore the Adversarial Domain Adaptation (ADA) technique and evaluate its performance in comparison to TL.
References
Isabela A Albuquerque, João Monteiro, Mohammad Darvishi, Tiago H Falk, and Toannis Milliagkas. General- ixing to unseen domains via distribution matching. arXiv preprint 1:Xix:1911.0000 0.00804, 2019.
Maurer Anderson, Cooper Hood, and Cook. Deep drilling basic research: volume 4-system description. final report. Technical report, Maurer Engineering Inc., Houston, TX and University of California, 1990.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprinl arXiv:1907.028)) 2019.
Mahsa Baktashmotlagh, Mchrtash T Harandi, Brian Lovell, and Mathicu Salzmann. Unsupervised do- main adaptation by domain invariant projection. In Proceedings of the IEEE international conference on computer vision, pages 769-776, 2013.
Theresa Baungartner and Eric van Oort. Pure and coupled drill string vibration pattern recognition high frequency downhole data. In SPE Annual Technical Conference and Exhibition, pages SPE 1775-17955. SPE, 2014.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances in newral information processing systems, 19, 2006.
Shai Ben-David. John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79:151-175, 2010.
Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22 (14):49--577, 2006.
Shai Ben David, Tyler Lu, Teresa Tau, and Dávid Pâl. Impossibility theorems for domain adaptation. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 129 136. JMLR Workshop and Conference Proceedings, 2010.
Behzad Elahifar and Erfan Hosscini. A new approach for real-time prediction of stick-slip vibrations en- hancement using model agnostic and supervised machine learning: a case study of norwegian continental shelf. Journal of Petroleum Exploration and Production Technology, 14(1):175 201, 2024.
23

---

Published in Transactions on Machine Learning Research (03/2025)
Tongtong Fang. Nan Lu, Gang Niu, and Masashi Sugiyama. Rethinking importance weighting for deep learning under distribution shift. Advances in neural informalion processing systems, 1911999-12077 2020.
Yuqi Fang, Pew-Thian Yap, Weili Lin, Hongtu Zhu, and Mingxia Liu. Source-free unsupervised domain adaptation: A survey. Neural Networks, page 106230, 2024.
Basura Fernando, Amaury Habrard, Marc Scbban, and Tinne Tuytelaars. Unsupervised visual domain adaplation using subspace aligmment. In Proceedings of the IEEE international conference on compuler vision, pages 1960-2967) 2013.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In Interna- tional conference machine learnóng, pages 1180 1189. PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-advvrssarial training of neural networks. Journal of machine learning reseorch, 17(39):1-33) 35, 2016.
Boqing Gong, Kristen Grauman, and Fei Sha. Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In International conference on machine learning, pages 222 230. PMLR, 2013.
Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. Domain adaptation for object recognition: An unsupervised approach. In 2011 international conference on compuler vision, pages 190-1 IEEE, 2011.
Chiranth Hegde, Harry Millwater, and Ken Gray. Classification of drilling stick slip severity using machine learning. Journal of Petroleum Science and Engineering, 179:1023 1036, 2019.
Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex Smola. Correcting sample selection bias by unlabeled data. Advances in neural information processing systems, 19, 2006.
Yunpei Jia, Jie Zhang, Shiguang Shan, and Xilin Chen. Singlo-side domain generalization for face anti- spoofing. Proceedings of the IEFE/CVP Conference on Computer Vision and Pallern Recognition, pages 8484 8493, 2020.
Daniel Kifer, Shai Ben-David, and Johannes Gehrke. Detecting change in data streams. In VLDB, volume 4, pages 180 191. Toronto, Canada, 2004.
Boung-Bum Kim, Karl Stratos, and Dongchan Kim. Adversarial adaptation of synthotic or stale data. In Proceedings of the 55th Annual Meeling of the Association for Compulational Linguistics (Volume 1: Long Papers), pages 1297-1307, 2017.
Dimitrios Kollias, Anastasios Arsenos, and Stefanos Kollias. Domain adaptation explainability & fairness in ai for medical image analysis: Diagnosis of covid-19 based on 3-d chest ct-scans. In Proceedings of the IEEE/CVP Conference on Computer Vision and Poltern Recognation, pages 4907 4914, 2024.
Hailin Li. Time works well: Dynamic time warping based on time weighting for time series data mining, Information Sciences, 547:592 608, 2021.
Jingjing Li, Zhiqi Yu, Zhekai Du, Lei Zhu, and Heng Tao Shen. comprehensive survey on source-free domain adaptation. JEFE Transactions on Pottern Analysis and Machine Intelligence, 46(8):5743-5700 2024.
Xiang Li, Wei Zhang, Hui Ma, Zhong Luo, and Xu Li. Domain generalization in rotating machinory fault diagnostics using deep neural networks. Neurocompuling, 403:409 000, 2020.
Zheng Li, Yun Zhang, Ying Wei, Yuxiang Wu, and Qiang Yang. End-to-end adversarial memory network for cross-domain sentiment classification. In IJCAT, pages 2237-222 2243, 2017.
24

---

Published in Transactions on Machine Learning Research (03/2025)
Mingxin Liu, Jiong Mu, Zitong Yu, Kun Ruan, Baiyi Shu, and Jie Yang. Adversarial learning and decomposition-basel domain generalization for face anti-spoofing. Pattern Recognilion Letters, 155:171- 177, 2022.
Bo-Qun Ma, He Li, Yun Luo, and Bao-Liang Lu. Depersonalized cross-subject vigilance estimation with adversarial domain generalization. In 2019 International joint conference on neural networks (IJCNN), pages 1-8. TEFE, 2019.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint or arXiv:0902.3430, 2009.
Toshihiko Matsuura and Tatsuya Harada. Domain generalization using mixture of multiple latent domains. In Proceedings of the AAAT conference artificial intelligence, volume 34, pages
2020.
Aakanksha Naik and Carolyn Rose. Towards open domain event trigger identification using adversarial domain adaptation. arXiv preprint arXiv:2055. 11355 2020.
Thomas Marc Richard. Self-encited stick-slip oscillations of drag bits. University of Minnesota, 2001.
Ramy Saadeldin, Hany Gamal, and Salaheldin Elkatatuy. Detecting downhole vibrations through drilling horizontal sections: machine learning study. Scientific Reports, 13(1),6204, 2023.
Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
Yuelin Shen, Zhengxin Zhang, Jie Zhao, Wei Chen, Mohammad Hamzah, Richard Harmer, and Geoff Down- ton. The origin and mechanism of severe stick-slip. In SPE annual technicol conference and ezhibition. OnePetro, 2017.
Prasham Sheth, Indranil Roychoudhury, Crispin Chatar, and José Celaya. A hybrid physics-based and machine-learning approach for stick/slip prediction. In IADC/SPE International Drilling Conference and Ezhibition. OnePetro, 2022.
Anthony Sicilia, Xingehen Zhao, and Seong Jae Hwang. Domain adversarial neural networks for domain generalization: When it works and how to improve. Machine Learning, 1 112(():2685-2777 2622-, 2023.
Peeyush Singhal, Rahee Walambe, Sheela. Ramanna, and Ketan Kotecha. Domain adaptation: challenges, methods, datasets, and applications. IEEE access, 11:6973-7093 -7020, 2023.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. Generalizing to unseen domains: survey on domain generalization. IEEE transactions on knowledge and data engineering, (5(8))8052-8072 2022.
Xingxing Weng, Yuchun Huang, Yanan Li, He Yang, and Shaohuai Yu. Unsupervised domain adaptation for crack detection. Automation in Construction, 153:104939, 2023.
Ruihao Xia, Chaoqiang Zhao, Meng Zheng, Ziyan Wu, Qiyu Sun, and Yang Tang, Cmda: domain adaptation for nighttime semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21572-21581 2023.
Brian Xu, Mitra Mohtarami, and James Glass. Adversarial domain adaptation for stance detection. arXiv preprint Xiv: 02401, 2019.
Hana Yahia, Thomas Romary, Laurent Gerband, Bruno Figluizzi, Florent Di Meglio, Stephane Menand, and Mohamed Mahjoub. Combining machino-learnigg and physics-basod models mitigate stick-slip real-time. In SPE/TADC Drilling Conference and Fahibition, page D0315022R003. SPE, 2024a.
25

---

Published in Transactions on Machine Learning Research (03/2025)
Hana Yahia, Thomas Romary, Laurent Gerbaud, Stephane Menand, and Mohamed Mahjoub. Real-time stick-slip mitigation using combined machine learning and physics based techniques. In International Petroleum Technology Conference, page D031S099R001. IPTC, 2024b.
Yang Zha and Son Pham. Monitoring downhole drilling vibrations using surface data through deep learning. In SEG Technical Program Expanded Abstracts 2018, pages 2101-2105. Society of Exploration Geophysi- cists, 2018.
Chao Zhao and Weiming Shen. Adversarial mutual anformation-guided single domain generalization network for intelligent fault diagnosis. IEEE Transaclions Industrial Informalics, 19(3):2909 2918, 2022.
Huailiang Zheng, Rixin Wang, Yuantao Yang, Yuqing Li, and Minqiang Xu. Intelligent fault identiiccation based on multisource domain generalization towards actual diagnosis scenario. IEEE Transactions on Industrial Electronics, 67(2):1293-133 1304, 2019.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Paltern Analysis and Machine Intelligence, 45(4):4396 4415, 2022.
Xiaohua Zhu, Liping Tang, and Qiming Yang. A literature review of approaches for stick-slip vibration suppression in oilwell drillstring. Advances in Mechanical Engineering, 6:967952, 2014.
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43-76, 2020.
26


Reviews:
[{"review_id": "SyezZOSdg6", "reviewer": "Reviewer_9GhK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The paper is mostly clearly written and easy to follow. Sufficient context is provided for the drilling SSI prediction as a problem setup, particularly in explaining the significance of the SSI and the challenges associated with predicting it in drilling operations .\n\n## Weaknesses\n- The paper presents mostly empirical results. The findings are somewhat expected, and there are very limited new insights regarding the domain generalization methods in general or specifically in the context of SSI prediction applications. The authors primarily focus on the adversarial domain generalization technique -- the only DG technique being evaluated and compared, which may restrict the comprehensiveness of the empirical study. \n- Only one 'traditional' domain generalization technique is being studied, which may limit the paper's depth and prevent the authors from making new discoveries. Perhaps, If more domain generalization techniques were examined, the authors could potentially conduct a comparison and evaluation of their effectiveness, leading to more interesting results or new insights.\n- Were there any important modifications required for applying the adversarial domain generalization approach to the SSI prediction task? Perhaps the authors could highlight some challenges that demonstrate this is not merely a straightforward application of existing methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents an empirical study on the effectiveness of domain generalization methods for predicting the Stick-Slip Index (SSI) in drilling applications. The authors compared one adversarial domain generalization approach to a standard baseline regression model, demonstrating that domain generalization techniques lead to a more robust overall predictor that generalizes well to prediction tasks from unseen wells. Additionally, the authors showed that incorporating few-shot fine-tuning during testing further enhances model performance, for both the baseline and DG, allowing the trained model to adapt effectively to the specific characteristics of data from unseen wells.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "Please kindly refer to the weaknesses above.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0vYHhyIJsZ", "reviewer": "Reviewer_ChHd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The paper is for the most part well-written in terms of readability, structure and presentation.\n- Assumptions under which DAN is effective is clearly described.\n- DAN outperforms the baseline model.\n- Data with similar characteristic are consistently used either training or test data, but not both, making efficacy of DAN more credible in the context of transfer learning.\n\n**Weaknesses**\n- Choosing the model from Yahia et al. (2024a) as the baseline is not justified based on literature review, i.e. (1) what are the alternatives to this choice as the baseline model and (2) why this particular model makes a good choice as the baseline model.\n- Missing analysis on to what extent the assumptions of DAN hold in this case.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Within the drilling industry, detection of stick-slip is of particular interest as it can significantly impact drilling performance and equipment integrity. Furthermore, the challenge of transfer learning to the target domain and training on diverse source domains with potentially disparate underlying distributions is present in the context of data gathered from different drilling wells. \n\nTo this end, this paper adapts the Domain-Adversarial Network (DAN) proposed in Ganin et al. (2016), using surface measurements to predict the stick-slip index (SSI), which is a metric quantifying the severity of a stick-slip. In particular, DAN aims to learn domain-invariant representations of surface measurements across different source distributions, i.e. data from different wells. This is realized by maximizing the discriminator's loss in the typical adversarial setting involving a generator of the representation of the input data and discriminator serving as proxy of how distinguishable different representations generated are. To clarify, the DAN considered in this paper consists of the following three networks:\n\n- Generator: Alternating LSTM cells with Layer Normalization (LN) layers, taking in surface measurements and outputs its representation.\n- Discriminator: Fully Connected Network (FCN), taking in representation and predicts the well the representation is originating from.\n- Predictor: FCN, taking in representation and predicts the corresponding SSI.\n\nThis model is compared against the baseline model from Yahia et al. (2024a), which the authors claim to have the same generator architecture but different predictor, although this predictor has identical loss function, metrics, and optimizer as that of DAN. I cannot verify the correctness regarding the statements about the baseline model due to the paywall present in accessing Yahia et al. (2024a) as it is published in a rather niche proceeding. See for questions regarding this in **Requested Changes**. \n\nAfter grid search for the hyperparameters of DAN, DAN appears to outperform the baseline model by around 10% on all test wells. Similarly, after fine-tuning the models on the target distribution, DAN still outperforms the baseline model.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Major**\n- I believe that it is essential to justify the choice of the baseline model via literature study. This is because if the baseline model chosen is deprecated for today's practice then comparison to it as evidence of superiority of proposed model is meaningless. Therefore, I would like to request incorporation of (1) **what are the alternatives to this choice as the baseline model** and (2) **why this particular model makes a good choice as the baseline model**.\n- I believe that it is essential to discuss the applicability of the assumptions mentioned for DAN to the surface measurement data in relation to SSI. Model reliability is essential and verifying its assumptions builds trust in its use in reality. I understand that this may be difficult without comprehensive analysis but in my opinion **it cannot be left blank** if the assumptions are mentioned in the first place.\n- Recall that I do not have access to Yahia et al. (2024a) in which the baseline model used in this paper is proposed as mentioned in **Summary Of Contributions**. For the baseline model, what is exactly its architecture? Is it also DAN? Furthermore, in section 4.2, it is mentioned:\n*\"The only addition is a final dense layer with a single neuron responsible for outputting the predicted SSI.\"*\n. However, for DAN, it is mentioned that *\"The SSI-predictor has a single output neuron responsible for predicting the SSI.\"*. To me, this means that the predictor for the baseline model is exactly the same as that of DAN. **My request is therefore to clearly state the difference between the two.**\n\n**Minor**\n- I am unsure about the second equality of Equation 2 on the expected risk. According to the change of variables, for the second equality to hold, $R'(x)P(R(x))=P(x)$, where $P$ is the probability measure on $D_T$. **This should be justified or mentioned** in the text unless I am miscalculating something.\n\n**Inessential suggestion**\n- Personally, I am not a big fan of this idea of making the representation agnostic to the source distribution via a discriminator. It is just unreliable to what extent the generator is able to identify the separating set in the causal graph or what one might call the invariant features. For this reason, I am curious if the authors have encountered any difficulty in training the DAN that reflects this problem.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "WgMHnVo4Rs", "reviewer": "Reviewer_QDMe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The submission introduces a novel domain generalization approach for predicting the Stick-Slip Index (SSI) using adversarial domain generalization. This method effectively addresses the challenge of generalizing across wells with distinct characteristics, which is often a major limitation in traditional machine learning models.\n\nThe submission has significant practical applications in the field of drilling and oil/gas exploration, where accurate prediction of stick-slip events is critical for optimizing drilling efficiency and preventing equipment damage.\n\n\nThe authors perform a grid search for hyperparameter tuning, ensuring that the model is optimized for performance. This level of detail in model selection improves the robustness of the results and contributes to the reliability of the proposed approach.\n\n----\n\nweakness\n\n\nThe paper does not present significant technical innovation. The use of adversarial learning and other techniques (such as domain generalization) are well-established methods already widely applied in Out-of-Distribution (OOD) problems. While these methods are certainly valuable, the paper does not demonstrate a novel approach or an innovative contribution that justifies its publication in TMLR, which typically focuses on papers offering new theoretical insights or novel algorithmic advancements.\nInsufficient Error Analysis:\n\nThe paper does not provide a detailed error analysis for the model's predictions. While performance metrics such as MSE and DTW are reported, there is a lack of thorough investigation into why the model fails in certain cases or how errors are distributed across the test set. An in-depth analysis of failure cases, such as data-specific errors, model limitations, or domain shifts, would provide valuable insights into how the model can be improved and in which scenarios it is less effective.\nIntroduction Section Needs Improvement:\n\nThe Introduction section is overly long and seems to blend the related work with the contribution of the paper, leading to a lack of focus. It feels more like a general literature review rather than a clear statement of the problem and novel contributions of the paper. The information density is low, and the key findings and novelty of the work are not highlighted effectively. A more concise and focused introduction would help set the context more clearly and emphasize the core research question and contribution.\nPotential Improvements with Stronger Adversarial Learning Strategies:\n\nIf the proposed adversarial learning approach is indeed effective, it would be useful to explore whether stronger adversarial techniques (such as Invariant Risk Minimization (IRM) or ELS [Free Lunch for Domain Adversarial Training: Environment Label Smoothing] could provide further performance improvements. While the current method shows some promise, incorporating or comparing with these more advanced techniques could enhance the effectiveness and robustness of the model.\n\n\nIt is noted that the reviewer has limited familiarity with the application area (e.g., drilling operations), which restricts the ability to assess the domain-specific contributions of the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, a new domain generalization method is proposed for predicting the stuck slip index (SSI) during drilling. SSI is a key index used to measure the strength of torsional vibration under the bit, and the stuck slip phenomenon in torsional vibration usually affects drilling efficiency and may even lead to equipment damage", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weakness", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper evaluates the performance of domain generalization techniques, specifically Adversarial Domain Generalization (ADG) and Invariant Risk Minimization (IRM), for predicting the Stick-Slip Index (SSI) in drilling operations, analyzing their suitability for time series regression tasks with limited data. 

The AE and reviewers acknowledged the paper's practical value. Given the empirical focus of this research, the authors are encouraged to make their code publicly available to support practical implementations.
Decision: Accept (poster)