You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


1. Introduction {#s0005}
===============

Post-translational modifications (PTMs) play a central role in the regulation of nearly all cellular processes. Among the twenty canonical amino acids, lysine residues undergo the most diverse range of modifications [@b0005], [@b0010]. For instance, positively-charged lysine residues can be acetylated, ubiquitinated, SUMOylated, glycated, butyrylated, succinylated, and methylated, with different types of modifications leading to different functional outputs in the cell. Recently, malonylation was discovered as yet another type of lysine PTM [@b0015]. Malonylation, which occurs in both eukaryotic and prokaryotic cells, involves a covalent linkage between the ε-amino group of lysine and a malonyl moiety. Unlike acetylation, which involves the addition of relatively small two-carbon chain, malonylation extends the lysine side chain by an additional 4 carbons, making it one of the bulkier acylation modifications. Moreover, because the malonyl moiety contains a negatively-charged carboxylic acid group at the γ-position, malonylation effectively switches the charge of the modified lysine residue from +1 in the unmodified state to −1 following malonylation [@b0015], [@b0020], [@b0025], [@b0030]. As a consequence, malonylation can have a profound influence on protein function. Indeed, several studies have found that malonylation influenced signaling processes in a variety of organisms, including humans [@b0020], [@b0035], mice [@b0035], [@b0040], and the bacterium, *Saccharopolyspora erythreae* [@b0045]. Likewise, malonylation has also been implicated in the regulation of inflammation and core metabolic processes [@b0050], [@b0055], [@b0060] as well as numerous pathophysiological disorders, such as cancer, cardiac ischemia and muscle weakness [@b0030], [@b0035].

Recently, several proteomics strategies have been developed to identify malonylation sites in cells, including mass spectrometry-based methods, chemical probes, affinity enrichment and label-free quantitative methods [@b0015], [@b0020], [@b0065], [@b0070], [@b0075]. However, these approaches are often time-consuming, expensive, and require a high level of technical expertise. As a consequence, several groups have recently developed computational methods to predict malonylation sites *in silico*. These methods, which complement existing experimental methods, not only have the potential to identify novel malonylation sites on proteins that might not be detected by experimental methods (e.g., low abundance proteins or proteins that are only modified under specific cellular conditions), but they also have the potential to offer new insights into the molecular characteristics that lead to malonylation. The first malonylation site predictor, Mal-Ly, was developed using a support vector machine (SVM) algorithm with feature selection of minimum redundancy/maximum relevance (mRMR) based on experimentally identified malonylation sites from *M. musculus* [@b0080]. Subsequent methods, including MaloPred [@b0085], SPRINT-Mal [@b0090], iLMS [@b0095] and the method of Xianget [@b0100], also employed SVMs with distinct feature sets to integrate malonylation site data from various organisms, including humans, mice, *S. erythraea* and *E. coli* [@b0085], [@b0090], [@b0095], [@b0100], [@b0105]. Likewise, several methods, such as LEMP [@b0110] and the method developed by Zhang et al. [@b0115], utilized ensemble methods to identify malonylation sites from diverse species. Notably, during the development of LEMP, which is the most recent and best performing malonylation site prediction method developed to date, Chen et al. integrated a deep learning (DL) network classifier based on long short-term memory (LSTM) with word embedding (LSTM~WE~) and RF with enhanced amino acid content (EAAC) features [@b0110]. Nonetheless, there is still room for improvement, as evidenced by MCCs of just 0.246 and 0.244 using 10-fold cross-validation and an independent dataset, respectively. Indeed, though great strides have been made in the performance of malonylation site predictors over the past several years, most successfull methods still suffer with respect to sensitivity (SN) and overall performance (as assessed by MCC). Therefore, we sought to develop malonylation site prediction tools based on DL and RF/Xgboost ([Fig. 1](#f0005){ref-type="fig"}). While a DL-based strategy is highly efficient and not dependent on manually extracted features, the use of RF/Xgboost permits identification of informative features along with different numbers of features. Our methods, which we termed DL-MaloSite and RF-MaloSite, respectively, each improved prediction of malonylation sites considerably and provide complementary information about malonylation site selection.Fig 1Schematic diagrams illustrating the architectures of RF-MaloSite (A) and DL-MaloSite (B).

2. Materials and methods {#s0010}
========================

2.1. Dataset and pre-processing {#s0015}
-------------------------------

The original dataset used in the development of our methods was retrieved from previous studies and contained experimentally identified malonylation sites from mice and humans [@b0040], [@b0065]. The training and independent sets were assembled in a manner similar to that described by Chen et al. [@b0110]. We also employed the same procedure outlined in Chen et al. [@b0120] and Olsen et al. [@b0125] to keep only non-redundant data. To this end, we used CD-Hit to remove homologous sequences that shared greater than 30% identity [@b0130]. Any sequences that contained "placeholder" resides, such as X, O, U, Z, or J were discarded. Briefly, we first defined experimentally-verified malonylation as positive sites and lysine residues not known to be malonylated as negative sites. We then took a certain number of residues upstream and downstream of the positive/negative site to define a window. To determine the optimal window size, different window sizes were tested and the window with the highest MCC was selected. Finally, we randomly split our dataset so that one-fifth of the dataset was set aside for the independent set and the remaining four-fifths were used for training. Summary statistics for the training and independent sets are given in [Table 1](#t0005){ref-type="table"}.Table 1The number of positive and negative sites in the training and independent sets before (left) and after (right) balancing:Original dataPositive sites (before/after)Negative sites (before/after)Training3978/397868,595/3978Testing988/98816,097/988

*Feature encoding.* To develop a computational tool to distinguish malonlaytion sites, we first needed to convert each residue in the protein sequences into a numerical value. Recently, several features have been proposed to convert protein primary amino acid sequences into mathematical expressions, including amino acid composition (AAC), pseudo amino acid composition (PSAAC), gene ontology (GO), position specific scoring matrix (PSSM), and various other physicochemical properties [@b0120], [@b0125], [@b0130], [@b0135], [@b0140], [@b0145], [@b0150], [@b0155]. Likewise, other studies have generated features based on structural information, such as secondary structure, super secondary structure, accessible surface area and local backbone angles [@b0160], [@b0165], [@b0170], [@b0175], [@b0180]. Each of these features can provide insights into distinctive types of posttranslational modification. There have been some ensemble approaches for bioinformatics problems [@b0185], [@b0190]. In this study, we extracted some features from the Features Extraction from Protein Sequence (FEPS) web application [@b0195]. In total, we extracted 4246 features from the FEPS server. We also combined these features with three types of physiochemical properties, such as amino acid properties (AAP), binary encoding (BE) and enhanced encoding features (EAAC). The final number of features was 6590. The feature classes, many of which are similar to those used to develop our glutarylation site predictor, RF-GlutarylSite, are summarized in [Table 2](#t0010){ref-type="table"} [@b0200]. A detailed description of each feature class can be found in [@b0200]. The other features are EAAC, amino acid index (AAI) derived from this study [@b0110], and AAP that has been implemented in prediction of S-sulfenylation sites [@b0205]. Here, we briefly describe EAAC.Table 2Composition of the complete feature set used during initial model development. Detailed descriptions of each feature class except EAAC, AAP, and AAI are provided in [@b0200].NOFeature ClassAbbreviationFeature Length1Binary featureBIN5602Amino Acid CompositionAAC203Composition of amino acid pairsAACP4004K-space amino acid pairsKSAAP24005Composition, Transition and DistributionCTD147 (C:21; T:21, D:105)6Conjoint triadCT5127Pseudo-amino acid compositionPseAAC478AutocorrelationA~u~7209Amino acid factorAAF14010Amino acid propertiesAAP392\*11Amino acid indexAAI812\*12Enhanced amino acid compositionEAAC440\*13Combined all featuresTotal6590

2.2. Enhanced amino acid composition {#s0020}
------------------------------------

EAAC, which was applied by Chen et al. during the development of LEMP [@b0110], is an enhanced version of amino acid composition that counts the frequency of sliding window across the peptide or sequences. Instead of counting the frequency of individual resides within a given fragment, EEAC starts upstream of the site and ends downstream. In our work, we calculated the occurrence of a sliding window that consisted of eight residues, yielding a final dimensional vector of 440.

3. Feature selection {#s0025}
====================

Though using a larger number of relevant features can provide clearer information about the target, if the feature set consists of many irrelevant or correlated features, classification performance can actually be diminished with an expanded feature set. Moreover, large numbers of features cause major issues for computational problems and can increase computational time. Therefore, it is often advantageous to reduce the number of irrelevant and redundant features when employing machine-learning methods [@b0210], [@b0215]. To this end, we employed the Gradient Boosted Trees method, Xgboost [@b0220], to detect non-linear associations from the features set. Xgboost has been applied to extract the highest valuable features in many studies [@b0200], [@b0225], [@b0230]. We applied Xgboost [@b0220] in Python with the Scikit-learn (v 0.19.0) package [@b0235] to capture significant attributes from our training set.

Essentially, we calculated Gini impurity for each feature to find the one that had the best split. This means that we reduced the impurity for each node and reached to the terminal node. High impurity values would suggest more uncertain in identifying class labels. It required more split until reaching to the leaf node.

Next, we computed information gain for each feature and found the one that had maximum values, which was used as important features for the first tree. The same procedure was repeated for other trees. After that, we determined the average importance of each feature from aggregated trees and recorded those as important features for our method.

Gini impurity can be described by:$$G = \sum_{m = 1}^{c_{n}}t_{m}{(1 - t_{m})}$$where C~n~ is the number of classes and t~m~ is the probability value of m. Operating values of each node in the gradient boosted trees, we computed Gini Importance by Gini Importance formula as follows:$$A = G_{parent} - G_{child1} - G_{child2}$$

Any attribute (A) with a significance value lower than 0.002 as threshold was considered as an unimportant attribute and was therefore removed from the feature set. We selected only 104 features as informative features for our approach whereas the rest of the attributes were rejected from the training set due to small importance impact.

*Random forest method*. RF is a supervised machine-learning technique that uses a bootstrapping algorithm [@b0240], [@b0245], [@b0250]. It has been broadly applied to many bioinformatics problems, such as those described in [@b0255], [@b0260], [@b0265]. The construction of a random forest is achieved by aggregation of decision trees. Each tree is a random subset of data points (d) from samples (S) with a random subset of attributes (A). It is assumed that data point (d) was randomly chosen with replacements from samples (S) with a random subset of attributes (A). Following that, the best select node was designated from A. Next, each decision tree was expanded as likely without trimming. Finally, using this information, each tree will assign positive or negative class, but the last categorization is made based on maximum voting from the entire tree. In this work, RF was employed using Python (v 3.6.0) with the Scikit-learn (v 0.19.0) [@b0235], and pandas (v 0.20.3) [@b0270], packages were executed to build our approach.

**Deep learning based approach**. One of the principle advantages of DL over traditional machine learning approaches is the ability to learn relevant features itself. The input for our DL approach, which we named 'DL-MaloSite', is the FASTA format sequence windows. A window size of 29 gave us the optimal results, so it was used in this study. First, the alphabetic sequence for each peptide was converted to respective integers from 0 to 20 for 20 amino acids. These integers were encoded with the window size 29, which were then fed into the DL network.

**Model architecture.** The overall DL architecture is shown in [Fig. 1](#f0005){ref-type="fig"}. The integer encoded input is passed to the embedding layer [@b0275]. Embedding dimension of 21 gave us the optimal results. At the beginning, weights are initialized randomly but the model learns gradually for the improved vector representation. Each vectorization is an orthogonal representation in another dimension, so commonly co-occurring items are together in the vector space, thus performing better than one-hot encoding used in [@b0280], [@b0285]. The output dimension is 29 × 21. A lambda layer was then used to add pseudo dimension before passing to 2-dimensional convolutional layer.

By using 2D convolutional layer, a filter size of 15 × 3 was possible, which in turn allowed us to include the central PTM site in every stride. The use of this filter size, along with the disabling of padding, allowed the model to be optimized for training time without compromising performance. To avoid overfitting, a relatively high cutoff of 0.6 was used. In this model, two convolutional layers, one maxpooling layer, a fully connected layer with two dense layers and an output layer were used. A rectified linear unit (ReLU) was also used as an activation function for all layers and Adam [@b0290] was used as the optimizer for our architecture. More information about the model and its important parameters are provided in supplementary materials.

**Transfer learning.** The performance of different DL architectures greatly relies -on the quantity of dataset, which adversely affects the outcome. To circumvent the problems posed by a limited dataset, we employed a transfer learning approach similar to that used during the development of DeepPhos [@b0285]. Specifically, a model developed for a similar task was reused and retrained on a new dataset. In our study, for the second dataset, we used a model trained on a mouse dataset as a base for training on a human dataset to improve the performance.

We have implemented a similar approach during the development of our DL model, DeepSuccinylSite [@b0295] for succinylation site prediction, with better performance metrics than previous machine learning models without the use of manually extracted features.

**Model assessment and performance metrics**. To assess the implementation of our approach and the current malonylation site predictors' methods, we evaluated our methods using two assessment techniques. The first technique was established by ten-fold cross-validation. Specifically, we divided our data into ten chunks, of which one was used for validation while the other nine were used for training. We then evaluated the performance for each model. This procedure was iterated ten times for 10 models by changing training data points and test data. Lastly, we computed the average results from each iteration. The second technique was using an independent test set to evaluate the overall quality of our approaches.

To evaluate the quality of RF-MaloSite and DL-MaloSite, four statistical metrics were adopted. These metrics were accuracy (ACC), sensitivity (SN), specificity (SP), and Matthew's correlation coefficient (MCC). They have been employed to assess the performance of many approaches in different works [@b0195], [@b0300], [@b0305].

These measurements can be computed as follows:$$ACC = \frac{TP + TN}{TP + TN + FP + FN} \times 100$$$$SN = \frac{TP}{TP + FN} \times 100$$$$SP = \frac{TN}{TN + FP} \times 100$$$$MCC = \frac{\left( {TP} \right)\left( {TN} \right) - {(FP)}{(FN)}}{{\sqrt{}}(TP + FP)(TP + FN)(TN + FP)(TN + FN)}$$

From these metrics, TP denotes true positives (i.e., correctly assigned malonylation sites), TN denotes true negatives (i.e., correctly assigned non-malonylation sites), FP represents false positives (i.e., non-malonylation sites that were incorrectly assigned as malonylation sites) and FN specifies false negatives (i.e., *bona fide* malonylation sites that were incorrectly assigned as non-malonylation sites). Since MCC reports proportions of TP, TN, FP and FN, the assessment value of MCC is generally reflected as alternate for complete quality of approach [@b0310], [@b0315], [@b0320], [@b0325].

Receiver Operating Characteristics (ROC) curves were adopted as an additional quality metric. The ROC curve is a graphical plot that shows the performance of sensitivity against 1 -- specificity with every likely cutoff [@b0330] while the area under curve (AUC) denotes the level or quantity of separability.

4. Results and discussion {#s0030}
=========================

To facilitate the computational prediction of malonylation sites in proteins, we sought to develop a series of malonylation site predictors based on machine learning strategies. To this end, we first established datasets for training and testing of the algorithms. The dataset used in this work was initially extracted from Chen's study [@b0110]. However, this dataset had a highly imbalanced number of positive and negative samples, with a ratio of approximately 16 negative sites for every positive site (i.e., a 1:16 positive-to-negative ratio). Previous studies suggest that imbalanced datasets can cause serious complications with machine learning algorithms, potentially leading to reduced accuracy of prediction [@b0335], [@b0340]. To address this potential complication, several computational methods have been described to balance datasets, including over-sampling and under-sampling [@b0260], [@b0345], [@b0350]. Here, we employed an under-sampling strategy [@b0355], which was a technique that randomly selected the same number of instances from non-malonylation sites that equally balanced with the size of instances of malonylation sites. This led to a 1:1 positive-to-negative ratio for the training sets used to establish our methods.

Once the training and testing sets were established, we next used the training set to evaluate several statistical algorithms, including SVM, RF, DL, neural network (NN) and k-nearest neighbor (KNN) methods. Initially, we integrated all 6590 features uniformly into these algorithms and compared the performance of each algorithm based on 10-fold cross-validation ([Tables S1 and S2](#s0055){ref-type="sec"}). These analyses suggested that inclusion of the entire feature set did not improve the methods' ability to distinguish between those sites that were malonylated and those that were not (and may have even hindered performance). This is likely due to the inclusion of a large number of redundant and non-associated variables. Because decision trees, such as RF and Xgboost, provide information about the relative contribution of each feature to overall method performance, we used Xgboost to identify those features that contributed most substantially to malonylation site prediction ([Figs. S1 and S2](#s0055){ref-type="sec"}; [Tables S3 and S4](#s0055){ref-type="sec"}). Based on these analyses, we selected 104 features with importance greater than 0.002. As can be seen in [Fig. 2](#f0010){ref-type="fig"}, a high percentage of the selected features came from pseudo-amino acid composition (PseAAC; 20%), enhanced amino acid composition (EAAC; 14%) and composition, transition and distribution (CTD; 14%). Indeed, all 10 of the most important features came from one of these three feature classes, with 5 of the 10 coming from CTD, 3 of the 10 coming from PseAAC and 2 of the 10 coming from EAAC ([Fig. 3](#f0015){ref-type="fig"}; [Table 3](#t0015){ref-type="table"}). Together, these data suggest that the physiochemical properties of the amino acids surrounding the modified lysine (and particularly the presence of positively charged residues) plays a major role in malonylation site selection. On the other hand, sequence-based features, such as composition of amino acid pairs (AACP) was the least important feature, exhibiting the lowest percentage among all of the important features.Fig 2The distributions of each kind of attribute for optimal features with the corresponding percentage score. The total number of selected attributes was 104.Fig 3Top ten most important features with corresponding weights.Table 3Top ten features ranked by information gain score. PseAAC: Pseudo-amino acid composition; EAAC: Enhanced amino acid composition; CTD: Composition, distribution and transition.RankFeatures TypeDescription1APAAC8PseAAC feature at a particular site2ChargeD1100Distribution feature denoting positive charge of class 1 at last position3SW.12.KEAAC feature denoting enrichment of lysine at portion 124PolarityC1Composition feature of class 1 for polarity property5ChargeD1001Distribution feature which denotes positive charge of class 1 at first position6APAAC1PseAAC feature at particular site7ChargeD1025Distribution feature which denote positive charge of class 1 at second location8APAAC20PseAAC feature at particular site9SolventAccessibilityC1Composition feature of class 1 for Solvent Accessibility10SW.15.KEAAC feature denoting enrichment of lysine at portion 15

Once determined, the 104 important features were integrated into the RF-, SVM-, NN- and KNN-based frameworks (since DL does not utilize feature information, it was not impacted by feature selection). This strategy helped to improve the performance of each algorithm substantially ([Table 4](#t0020){ref-type="table"}). Not only did feature selection help to simplify and focus the models, but it also has the added benefit of reducing computational time. Importantly, it also suggested that DL- and RF-based methods performed the best with respect to ACC, SN, SP and MCC. Similar results were observed when the independent set was used for evaluation ([Table 5](#t0025){ref-type="table"}) Therefore, we focused on these classifiers during subsequent model development.Table 4Comparison between various machine learning algorithms based on 10-fold cross-validation using 104 features. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.ClassifierACC (%)SN (%)SP (%)MCC**DL7582690.51RF7179630.42**SVM6471580.30KNN6167560.23NN6973650.37Table 5Comparison between our methods and various machine learning algorithms based on an independent test set using 104 features. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.**ClassifierACC (%)SN (%)SP (%)MCCDL7480680.49RF7076630.40**SVM6569620.29KNN6266580.23NN6876580.35

In addition to a balanced dataset and feature selection, the window size used during method development is also an important parameter in prediction quality. Indeed, when using the primary amino acid sequence as input, evaluation of the window size helps to identify the impact that contiguous residues have on a given method's ability to distinguish between positive and negative sites. Therefore, to determine the window size(s) that yielded the best performance, we examined window sizes ranging from 25 to 33 residues. During these analyses, we used MCC based on 10-fold cross-validation as a surrogate of overall method performance. While examination of the optimal window size for the DL method entailed a simple sliding window strategy with no feature selection, during the evaluation of the RF algorithm, every proposed window size used the optimal features that had been selected by Xgboost. As shown in [Table S5](#s0055){ref-type="sec"} and [Fig. S3](#s0055){ref-type="sec"}, the optimal window size for both the RF- and DL-based algorithms was found to be 29. For instance, the MCC of the RF method peaked at 0.42 using a window size of 29, suggesting that this window size contains more useful information for malonylation site identification than other windows. It is interesting to note that both the RF- and DL-based methods showed the best performance using this window size. This may suggest that 29 residues (i.e., 14 residues on each side of a central lysine residue) represents the preferred peptide length for malonylation to occur (and/or to prevent de-malonylation by Sirtuin family members). However, the molecular basis for this observation remains to be determined.

While both the final DL- and RF-based methods, which we termed DL-MaloSite and RF-MaloSite, respectively, performed comparably well with respect to ACC, SN, SP and MCC, in general, DL-MaloSite outperformed RF-MaloSite in malonylation site identification. The same was true based on AUC and precision recall ([Figs. S4](#s0055){ref-type="sec"} and [4](#f0020){ref-type="fig"}).Fig 4Precision recall (PR) curve based on the independent test set for DL-MaloSite (red) and RF-MaloSite (blue). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

5. Comparison between our methods and existing methods {#s0035}
======================================================

To determine how DL-MaloSite and RF-MaloSite compare to existing malonylation site prediction methods, we conducted a series of side-by-side comparisons between our methods and existing malonylation site prediction methods. During these analyses, we restricted our comparisons to datasets used during the development of the existing methods, which in some cases required us to implement transfer learning for DL-MaloSite. For instance, for comparisons between our methods and SPRINT-Mal [@b0090] and iLMS [@b0095], we used the same training and independent datasets used by Taherzadeh et al. [@b0090]. Because these datasets were organism-specific (i.e., from *H. sapiens*, *M. musculus* and *S. erythraea*) and DL-MaloSite was trained on a mouse dataset that contained a comparatively larger number of malonylation sites than the *H. sapiens* or *S. erythraea* datasets, we implemented transfer learning for DL-MaloSite when comparisons were made using datasets from these organisms. To this end, the same model was used as the initial phase to be retrained against the new dataset. The evaluations using 10-fold cross-validation against the mouse dataset suggest that, while SPRINT-Mal outperforms our methods with respect to ACC and SP by an average of 16% and 21%, respectively, DL-MaloSite and RF-MaloSite each achieved higher SN scores than the other methods. Indeed, the SN scores exhibited by DL-MaloSite and RF-MaloSite were \~45% and \~47% higher than those achieved by SPRINT-Mal and iLMS ([Table 6](#t0030){ref-type="table"}). As a consequence, DL-MaloSite and RF-MaloSite exhibited the highest MCC scores of all of the methods tested, with MCC scores that were 54% and 38% higher than those achieved by iLMS, respectively (and 90% and 71% higher than those achieved by SPRINT-Mal, respectively). Similar trends were observed using the independent dataset from mouse, where DL-MaloSite and RF-MaloSite achieved MCC scores that were 50% and 19% higher than those exhibited by iLMS, respectively (and 95% and 55% higher than SPRINT-Mal's MCC score) ([Table 7](#t0035){ref-type="table"}). The gains in MCC by our methods were likely due to improved TP rates, which led to substantially higher SN scores without sacrificing SP and ACC, where SPRINT-Mal achieved the highest scores.Table 6Comparison between SPRINT-Mal [@b0090], iLMS [@b0095], RF-MaloSite and DL-MaloSite based on 10-fold cross-validation using a M. musculus dataset. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesOrganismACC (%)SN (%)SP (%)MCCAUCSPRINT-Mal [@b0090]*M. musculus***80**49**81**0.210.74iLMS [@b0095]--49800.260.74RF-MaloSite68**72**65**0.360.75**DL-MaloSite70**71**68**0.40**0.73Table 7Comparison between SPRINT-Mal [@b0090], iLMS [@b0095], RF-MaloSite and DL-MaloSite based on an independent test set from M. musculus. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesOrganismACC (%)SN (%)SP (%)MCCAUCSPRINT-Mal [@b0090]*M. musculus***90**33**92**0.20**0.74**iLMS [@b0095]------0.260.72RF-MaloSite65**65**65**0.31**0.72DL-MaloSite68**85**51**0.39**0.72

The comparisons based on 10-fold cross validation using the human dataset showed that, while the SP scores exhibited by iLMS outpace those of DL-MaloSite and RF-MaloSite by 23% and 29%, respectively, our methods achieved SN scores that were 75% and 48% higher than those scored by iLMS[2](#fn2){ref-type="fn"} ([Table 8](#t0040){ref-type="table"}). Likewise, DL-MaloSite performed the best with respect to MCC using the human dataset, where it achieved MCC scores that were 2.2-fold higher than iLMS and 47% higher than RF-MaloSite. Similarly, performance evaluation using an independent dataset from humans suggests that, though SPRINT-Mal achieved the highest SP and ACC scores, DL-MaloSite and RF-MaloSite performed well with respect to all metrics (particularly SN and MCC) ([Table 9](#t0045){ref-type="table"}). For instance, the SN scores observed for RF-MaloSite and DL-MaloSite were 88% and 2.2-fold higher than that achieved by SPRINT-Mal. Likewise, DL-MaloSite exhibited the highest MCC score using the independent dataset from humans (0.39), followed by RF-MaloSite (0.24) and then by SPRINT-Mal (0.20).Table 8Comparison between SPRINT-Mal [@b0090], iLMS [@b0095] RF-MaloSite and DL-MaloSite based on 10-fold cross-validation using a H. sapiens dataset. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesOrganismACC (%)SN (%)SP (%)MCCAUCSPRINT-Mal [@b0090]*H. sapiens*----------iLMS [@b0095]--48**80**0.230.74RF-MaloSite6771620.340.74DL-MaloSite (Transfer Learning)**7584**65**0.500.78**Table 9Comparison between SPRINT-Mal [@b0090], iLMS [@b0095] RF-MaloSite and DL-MaloSite based on an independent test set from H. sapiens. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesOrganismACC (%)SN (%)SP (%)MCCAUCSPRINT-Mal [@b0090]*H. sapiens***91**35**89**0.200.70iLMS [@b0095]----------RF-MaloSite6266590.240.66DL-MaloSite (Transfer Learning)69**78**60**0.390.75**

The differences between method performance were even more pronounced when the independent dataset from *S. erythraea* was used for comparison. Indeed, though the overall trends were similar to those observed using datasets from the other organisms, SPRINT-Mal outperformed RF-MaloSite and DL-MaloSite by 73% and 96%, respectively, with respect to SP and by 32% and 26%, respectively, with regard to ACC ([Table 10](#t0050){ref-type="table"}). Conversely, DL-MaloSite and RF-MaloSite achieved SN scores that were a remarkable 3.9- and 3.4-fold higher than those exhibited by SPRINT-Mal. This culminated in MCC scores for DL-MaloSite and RF-MaloSite that were 3.3- and 2.67-fold higher than the scores observed for SPRINT-Mal.Table 10Comparison between SPRINT-Mal [@b0090], iLMS [@b0095] RF-MaloSite and DL-MaloSite based on an independent test set from S. erythraea. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesOrganismACC (%)SN (%)SP (%)MCCAUCSPRINT-Mal [@b0090]*S. erythraea***86**23**92**0.120.64iLMS [@b0095]----------RF-MaloSite657753**0.320.67**DL-MaloSite68**90**47**0.400.75**

Finally, we compared our methods to LEMP, the current gold-standard for malonylation site prediction [@b0110]. For these analyses, we used the training and independent test sets described by Chen during the development of LEMP. Since these datasets combined malonylation sites from human, mouse and bacteria, we did not need to employ transfer learning under these circumstances. Based on 10-fold cross-validation, the ACC scores obtained for RF-MaloSite and DL-Malosite were 19% and 15% lower than those achieved by LEMP, respectively ([Table 11](#t0055){ref-type="table"}). Likewise, the SP scores obtained by RF-MaloSite and DL-MaloSite were 31% and 24% lower than those exhibited by LEMP, respectively. In contrast, the SN and MCC scores for both RF-MaloSite and DL-MaloSite were markedly higher than those achieved by LEMP. For instance, the SN scores for RF-MaloSite and DL-MaloSite based on 10-fold cross-validation were 89% and 97% higher, respectively, than the SN score observed for LEMP. Importantly, the MCC scores for RF-MaloSite and DL-MaloSite were 75% and 113% higher, respectively, than that achieved by LEMP.Table 11Comparison between DL-MaloSite, RF-MaloSite and LEMP based on 10-fold cross-validation. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesACC (%)SN (%)SP (%)MCCAUCLEMP [@b0110]**88**42**91**0.24**0.82**RF-MaloSite7179630.420.77DL-MaloSite75**82**69**0.51**0.81

Similar results were obtained using the LEMP independent dataset, with RF-MaloSite and DL-MaloSite outperforming LEMP with respect to MCC by 1.6- and 2.0-fold, respectively ([Table 12](#t0060){ref-type="table"}). Likewise, the SN scores exhibited by RF-MaloSite and DL-MaloSite were 72% and 82% higher than the SN score achieved by LEMP.Table 12Comparison between DL-MaloSite, RF-MaloSite and LEMP based on the independent test. ACC: Accuracy; SN: Sensitivity; SP: Specificity; MCC: Matthew's Correlation Coefficient.FeaturesACC (%)SN (%)SP (%)MCCAUCLEMP [@b0110]**87**44**90**0.24**0.82**RF-MaloSite7076630.400.75DL-MaloSite74**80**68**0.49**0.81

Taken together, these the side-by-side comparisons with LEMP suggest that both RF-MaloSite and DL-MaloSite are able to identify malonylation sites with higher sensitivity and higher MCC scores than LEMP. In contrast, though their SP and ACC scores were decent, our methods did not perform as well as LEMP with respect to SP and ACC.

More broadly, we observed that both of our methods performed better than existing methods in predicting malonylation sites, as evidenced by their relatively high SN and MCC scores. In contrast, our methods had some challenges determining non-malonylation sites (i.e., TNs in our datasets). This may be a function of our under-sampling strategy or a consequence of the potentially large number of as-yet-unidentified malonylation sites. There are ways to increase the performance of our methods, especially with respect to the SP and accuracy metrics. For instance, additional structural features can be introduced to help the classifiers distinguish between malonylation and non-malonylation sites. Likewise, as the number of experimentally-determined malonylation sites grows, we can also increase the number of samples, which can help our method learn more about the problem and enhance prediction.

6. Conclusions {#s0040}
==============

Here, we described the development of two independent machine-learning methods designed to predict malonylation sites in proteins based on their primary amino acid sequences. Our methods, which we termed 'Rf-MaloSite' and 'DL-MaloSite', outperformed existing approaches with respect to SN and MCC. For RF-MaloSite, this highlighted the power of using important features generated via the Xgboost technique and integrated into our RF algorithm. Interestingly, our DL-based model, DL-MaloSite, was able to perform as well or better than existing methods without the need of manual feature extraction. This reduces hassle for manual feature extraction, which can introduce bias as well. For further improvements in our DL model, we can use multi-windows input in a manner similar to DeepPhos [@b0285]. We can also add feature information and position specific scoring matrices (PSSM) in addition to that of the current embedding encoded input to improve performance. However, the main challenge would be the size of data, which will likely be remedied as the number of experimentally-verified malonylation sites continues to grow. Our method can be beneficial for the signaling community and biologists interested in understanding and exploring the impact of malonylation sites on physiological and pathophysiological states. Likewise, these methods will help researchers explore crosstalk between malonylation and other similar types of lysine PTMs, such as acetylation, glutarylation and succinylation.

CRediT authorship contribution statement {#s0045}
========================================

**Hussam AL-barakati:** Methodology, Software, Writing - original draft. **Niraj Thapa:** Methodology, Software, Writing - original draft. **Saigo Hiroto:** Conceptualization, Writing - review & editing. **Kaushik Roy:** Conceptualization, Writing - review & editing. **Robert H. Newman:** Conceptualization, Writing - review & editing. **Dukka KC:** Conceptualization, Writing - review & editing, Supervision.

Declaration of Competing Interest
=================================

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data {#s0055}
==============================

The following are the Supplementary data to this article:Supplementary data 1

Supplementary data 2

This work was supported by National Science Foundation (NSF) grant nos. 2021734, 1564606 and 1901793 (to DK). RHN is supported by an HBCU-UP Excellence in Research Award from NSF (1901793) and an SC1 Award from the National Institutes of Health National Institute of General Medical Science (5SC1GM130545). HS was supported by JSPS KAKENHI Grant Numbers JP18H01762 and JP19H04176.

SPRINT-Mal was not evaluated by 10-fold cross-validation using the human dataset.

Supplementary data to this article can be found online at <https://doi.org/10.1016/j.csbj.2020.02.012>.

[^1]: Equal first author.