BACKGROUND
dna methylation is an important epigenetic mechanism modifying mammalian genomes.
it plays a major role in several biological phenomena, such as x chromosome inactivation, imprinting, regulation of gene expression, development, cell differentiation, and the onset and progression of multiple diseases.
the predominant form is cytosine methylation  at cpg dinucleotides.
these cpg sites are located nonrandomly in the genome, tending to occur within high density clusters of cpgs .
around 70 % of all cpg dinucleotides are methylated .
cpg islands constitute around 60 % of human promoters and are predominantly unmethylated, while the sites in the remaining 40 % are often hypermethylated  <cit> .
dna methylation patterns are known to be tissue specific  <cit> , and it has been shown that one of the effects of dna methylation is to contribute to transcriptional silencing, where proteins bind directly to methylated dna and recruit co-repressor complexes  <cit> , triggering the formation of repressive chromatin.
there are several studies that show that changes in dna methylation are not confined to early development, but rather occur over the entire life span of an organism, resulting in distinct age-related methylation profiles .
while this global ‘epigenetic clock’ correlates strongly with chronological age and could thus be used to evaluate or exclude age-related factors in analyses of neurodevelopmental or neurodegenerative disorders, the specific genes or networks that depend on this clock for their regulation remain largely unknown.
since the changes in methylation patterns are complex and likely involve multiple genes in a combinatorial manner, in this study our goal was to explore how a non-linear multivariate machine learning approach, capable of analyzing multiple cpg sites simultaneously, would interpret the data without any a priori hypothesis such as the direction or trajectory in which the changes in methylation occur.
we thus applied a rule-based approach to a public methylation dataset profiled from the prefrontal cortex of the brain  <cit> , for which we first examined changes across all age boundaries.
after applying monte carlo feature selection  <cit>  to rank the cpg sites by significance, we identified five distinct age groups, with marked transitions between them.
we then used rosetta  <cit> , which implements rough sets theory  <cit> , to construct rule-based models based on the identified cpg loci.
methods
data preprocessing
the data set used in this work, numata et al.
<cit> , comprises dna methylation data from  <dig> samples, taken from individuals ranging from fetal to 84 years old, designed to study the dependence of methylation on age and gender.
genomic dna has been taken from dorsolateral prefrontal cortex.
illumina’s infinium humanmethylation <dig> beadchip was used to profile the dna methylation level at  <dig>  cpg dinucleotides.
we removed sites from the dataset if they fulfilled one or more of the following conditions:  cpg sites fall on chromosome x;  potentially nonspecific or polymorphic probes present on infinium humanmethylation <dig> beadchip; or  cpg sites with standard deviation of beta values <  <dig>  to remove uninformative sites.
beta values, which were measured from a population of cells and are therefore reported as average on a scale from  <dig> to  <dig>  were discretized into:  unmethylated if the chip reports a beta value of  <dig>  or lower;  methylated if the beta value is  <dig>  or higher; and  intermediate if the beta value is between  <dig>  and  <dig> .
discretizing the beta values was motivated by bibikova, le, barnes et al.
<cit> , who divided the beta values into the three groups “methylated”, “hemimethylated”, and “unmethylated”, proposing the threshold values  <dig>  and  <dig>  based on the overall distribution of beta values .
decision tables and selecting significant cpg sites
we constructed decision tables as follows : each row represents a sample with the values of the characterized features of that sample in the columns.
here, features are the selected cpg sites with their methylation levels as measured by the chip.
the last column holds the decision class the sample belongs to.table  <dig> a fragment of a decision table
the rows show the samples and the columns show the value for each property .
the last column contains the decision  to predict
we constructed  <dig> separate binary decision tables  by iteratively dividing the samples into two groups:  younger than a given age, or  older than or equal to a given age.
since there were very few samples older than  <dig>  we stopped the cuts at the age of  <dig>  in order to find the cpg sites that significantly contribute to classifying the samples to the age classes, we applied monte carlo feature selection   <cit>  to compute a normalized relative importance  score for each feature.
mcfs compensates for any imbalanced number of objects in each class.
constructing classifiers
figure  <dig> illustrates a schematic overview of the method.
the significant cpg sites for each age were extracted from the relevant decision table.
since the number of samples in different age classes was different, we ran a 100-fold under-sampling to avoid the bias of classification towards classes with more samples.
in under-sampling, a new decision table is created, consisting of the samples of the smallest group, plus a randomly selected subset of samples  from the other groups.
this process was repeated  <dig> times.fig.
<dig> summary of the analytical processes
we then created rule-based models using rosetta  <cit>  for each under-sampled decision table, and combined the results into one model for each age class.
rosseta implements rough set modeling  <cit> , the models of which are output in the form of human-readable if-then rules, which describe the relation between the cpg sites and the decision.
the rules are either conjunctive, consisting of a conjunction of conditions, such as:
if cg26158194=methylated and cg12078929=intermediate
then ‘olderthan50’
accuracy= <dig> , support=21
or singleton, having only one condition, such as:
if cg10608341=unmethylated then ‘youngerthan0’
accuracy =  <dig>  support = 30
each rule can be read as: if the condition in the if part is  satisfied for a sample, the model predicts it to be a member of the decision class .
the reported accuracy shows the ratio of correctly classified objects to all classified objects to a specific class, based on the model confusion matrix.
the support value shows the number of samples that satisfy the condition .
rosetta outputs a list of rules that are independent of each other, i.e.
if multiple sites, or combinations thereof, act as “predictors”, it reports all of them.
this, in turn, allows for interpreting the set of cpg sites used in the rules as being subject to changes in methylation at each specific age.
to avoid over-fitting of the data, we computed the statistical significance for each rule using a hyper-geometric distribution and bonferroni-correction, discarding rules with p >  <dig> .
we used the web based tool ciruvis  <cit>  to identify interactions among rule conditions, i.e.
rules that use the same cpg sites, ranked by strength - the sum of accuracies multiplied by support for all the rules in which it appears.
discretization into age groups
we computed the jaccard distance between all two-class decision tables based on the number of overlaps  between the significant features obtained for each individual two-class decision table, i.e.
given two significant features for age sets sfai and sfaj for decision classes i and j, the distance is computed as:
distance =  <dig> – /)
annotation of sites and rules
we annotated the cpg sites using annovar  <cit> , allowing for identifying the genomic region in which a cpg site was located, using the tags exonic, intronic, utr <dig>  utr <dig>  intergenic, splicing , and upstream .
functional annotation for the genes and the biological processes they are involved in was obtained from genecards .
RESULTS
informative cpg sites
in the numata et al.
dataset  <cit> , methylation levels measured by the illumina infinium humanmethylation <dig> beadchip are reported as beta values, i.e.
the methylation averaged over the cell population that was sampled, at  <dig>  cpg dinucleotide sites.
of these, we first removed  <dig> sites located on chromosome x to avoid sites in pseudo-autosomal regions on chromosome y in male participants.
we next removed  <dig> polymorphic loci, and  <dig> loci located on non-uniquely mapped probes .
for the remaining sites, we computed the standard deviation of beta values, and discarded sites that showed little variation in methylation throughout age, by applying a cutoff of  <dig>  , resulting in around  <dig>  cpg sites.fig.
<dig> histogram of standard deviation  of beta values for all the cpg sites after discarding the sites located on the x chromosome or located on nonspecific or polymorphic probes
discerning cpg sites over age
for any given age, we next binned the samples into two categories, either below or above this age, and repeated the process  <dig> times for ages  <dig> to  <dig>  we then applied monte carlo feature selection  to the respective two-class decision table to determine the set of cpg sites that significantly  contribute to the classification.
the number of such sites decreased as age increases , indicating that fewer sites are able to discern between samples at higher age and implying that changes in methylation are generally more pronounced at early ages and decrease with age.fig.
<dig> the number of significant cpg sites plotted for all two-class decision tables
we next examined the union of all significant cpg sites,  <dig> in total , for functional categories.
in this file, the coding is “0” for the sites not to be significant to discerning the age category, and “1” for the sites that are significant, marking genes also found by numata et al.
<cit>  in bold.
genes overlapping with these sites are involved in various developmental, proliferation and differentiation processes, but also in brain or neuron specific functions and diseases, as expected.
a total of  <dig> cpg sites only contributed to the classification of age  <dig> , consistent with the trend that was reported in the original study  <cit> .
the specific genes associated with these include artn, kmo, kcna <dig>  snrk, slc1a <dig>  haao, fev, chrd, spon <dig>  hesx <dig>  otof, elavl <dig> and sult1c <dig>  which are involved in brain and/or neuron specific processes, as well as slc1a <dig>  which is implicated in schizophrenia and other mental illnesses .
the site annotated to a 5′utr region of this gene showed an increase of methylation level from intermediate to methylated as age increases.
a site that changed from unmethylated in fetus to intermediate in adulthood was located upstream of fev, a gene implicated in the pathophysiology of psychiatric disorders such as depression, anxiety and eating disorders.
another site showing the same pattern is located in an intron of spon <dig>  which codes for a protein promoting adhesion and outgrowth of hippocampal embryonic neurons, as well as a site in the 5′ untranslated region of hesx <dig>  which codes for a transcriptional repressor protein in the developing forebrain.
we further identified genes involved in the regulation of processes such as cell migration, cell differentiation and cell progression, namely s100a <dig>  arhgap <dig> and lims <dig>
there were  <dig> sites exclusive to classifying age  <dig>  which were associated with the genes msx <dig>  myl <dig>  mpg, actn <dig>  nags, dydc1-dydc <dig>  scn4b, hydin, tst, lft, ctsz, emp <dig> and net <dig>  there were fewer sites involved in classifying older ages , located in genes such as ank <dig>  hoxa <dig>  tmem <dig>  atp8a <dig>  tex <dig> and tinagl <dig>  several cpg sites were involved in classification of a wide range of ages, some of which covered well-defined intervals.
for example, the sites associated with nptx <dig> , chrnb <dig> , rab <dig>  and esrp <dig>  were reported as significant for all ages between  <dig> and  <dig>
classification into age intervals
using the jaccard distance as a measure for the similarity between the calculated significant cpg sites for each age above  <dig>  we computed a full distance matrix and applied hierarchical clustering in r .
there are three distinct groups comprising age  <dig> to  <dig>  age  <dig> to  <dig>  and age  <dig> to  <dig>  with the latter exhibiting sub-groups within their respective clades.
the overall topology also reflects the trend observed earlier, in that age groups become broader with increasing age, following this clustering, we constructed a four-class decision table based the classes: ‘fetus’, ‘age 0–4’, ‘age 5–27’ and ‘age28plus’.
mcfs identified  <dig> significant cpg sites on this set, on which we trained a classifier using rosetta.
the resulting model had an accuracy mean of 90 % , as measured by 10-fold cross-validation.fig.
<dig> hierarchical clustering of the significant cpg sites obtained from running mcfs on all two-class decision tables.
the distance between every two significant cpg sets is based on the number of overlapping sites
analysis of the rule models
the model consisted of a set of rules that classify samples to a decision class according to the values of the significant features .
for further analyzing these rules, we first filtered by significance , accuracy , and support .
the top five most significant rules for each of the age classes are listed in additional file  <dig>  we tested the sites in each rule against a multivariate linear regression model, and highlighted the rules not significant in the regression in red.
as expected, all singleton rules show significance in linear regression, whereas more than half of the conjunctive rules do not.
table  <dig> lists the confusion matrix, indicating that the classification power for fetus and age 0– <dig> is higher than the other groups.
the fetus class comprised all of the singleton rules , which is consistent with marked methylation level changes at birth, where single cpg sites are indicative for this transition .
by contrast, the majority of rules classifying the older age groups comprise were conjunctive, i.e.
containing a combination of sites .table  <dig> confusion matrix for the rule-based classifier with four decision classes.
the number of samples in each class is shown in parenthesis
fig.
<dig> plot of the beta values of all samples for cg <dig> annotated to the upstream of fev that classifies fetus samples.
the red line shows the margin between unmethylated and intermediate beta values.
if cg24178740 = unmethylated then ‘fetus’.
support is  <dig> and accuracy equals to  <dig> as all the objects classified as fetus turn out to be fetus
fig.
<dig> plot of the beta values for two cpg sites appearing in the top significant rule for classifying age51plus: if cg27554782 = intermediate and cg21053323 = intermediate then ‘age51plus’.
accuracy equals to  <dig>  and support equals to  <dig> for this rule.
the red lines show the margin between unmethylated and intermediate beta values
while the classifier for age 0– <dig> contained conjunctive rules with perfect classification power on the data set, the rule accuracy dropped for both the age 5– <dig> and age 28+ classes, while the number of conjuncts increased, indicating that the changes in methylation become less dramatic and have thus less power to discern the samples by learning from their methylation patterns.
when examining the direction in which methylation changes, we found that 81 % of the fetus rules showed a pattern of lower-to-higher methylation when transitioning into the next group.
the associated cpg sites were linked to genes involved in cellular development, growth and proliferation, and neuronal related pathways and mechanisms.
for example, the site , located within the tgfbr <dig> gene, whose decreased expression has been observed in various cancers  <cit> , showed a methylation level increase from below  <dig>  in fetus to  <dig>  in adulthood.
a similar pattern was observed for cg <dig>  upstream of artn, which is a member of the glial cell-derived neurotrophic factor family of ligands, as was the site cg <dig> upstream of tp53i <dig>  a gene induced by the tumor suppressor p <dig>  which increased in methylation levels to  <dig>  in adulthood .
sites showing the reverse pattern were associated with plek, pik3c2b, rd <dig>  kcna <dig>  acap <dig> and gpr37l <dig> genes.table  <dig> list of  <dig> significant singleton rules classifying fetus samples
all of these rules are perfect rules with accuracy of  <dig> and support of  <dig>
the cg <dig> site was annotated to the upstream region of nptx <dig>  this gene encodes neuronal pentraxin ii , which is involved in neuritic outgrowth, synapse remodeling and the aggregation of neurotransmitter receptors at synapses  <cit> .
the nptx <dig> gene is reported to be hypermethylated in tumors, including brain tumors  <cit>  and in this data showed an increasing pattern from  <dig>  towards  <dig>  from fetus towards age 5– <dig> class and up to  <dig>  in the age 28+ class.
the site annotated to fbxo <dig> also appeared in the rules with a similar pattern.
fbxo <dig> is related to alzheimer disease, as it regulates app processing  <cit> .
rules interactions
we used ciruvis  <cit>  to investigate the combinations of the features in the rules.
figure  <dig> depicts the outcome for each of the decision classes.
two features  are connected inside the circle if they co-occur in multiple rules.
the connections are shown as edges between the nodes.
the width and color of the edges reflect the connection score .
table  <dig> lists the two strongest interactions for all classes.fig.
<dig> top  <dig> strong interactions among the cpg sites for the rules, generated by ciruvis, for fetus , age 0– <dig> , age 5– <dig>  and age 28+ classes.
the outer ring lists the cpg sites and the inner ring shows the sites connected  to each of them by an edge.
the thicker and darker the edge, the stronger the interaction
some of these interactions indicate coordinated age-related changes in methylation of genes involved in the same biological function or process.
for example, gene tgfbr <dig> encodes tgf beta-receptor iii  that serves as a co-receptor for other types of tgf beta-receptors.
conversely, ectodomain shedding of tgfbr <dig> produces soluble tgfbr <dig>  which inhibits tgf beta signaling  <cit> .
in either case, changes in the expression of tgfbr <dig> are likely to affect the signaling through the tgf beta pathway.
the other gene listed as interacting with tgfbr <dig>  namely cdkn2b, encodes cyclin-dependent kinase inhibitor 2b, which is a potent inhibitor of the cell cycle.
in epithelial cells, cdkn2b is known to be induced by tgf beta  <cit> .
the link between tgfbr <dig>  cdkn <dig> and the cell cycle is likely relevant to the aging brain, as tgf beta stimulates proliferation of microglia  <cit> , brain resident macrophages, which play important role during aging.
another strong interaction lists gene tp <dig> together with hapln <dig>  gene tp <dig> encodes the tumor protein  <dig>  a member of the p <dig> family of transcription factors, with a role in neuronal differentiation and hippocampal development  <cit> .
its deficiency results in impaired self-renewal and premature differentiation of mouse neuronal progenitors  <cit> .
tp <dig> is also a major survival factor for postmitotic neurons  <cit> .
gene hapln <dig> encodes brain-specific hyaluronyan and proteoglycan link protein  <dig>  this protein in the cerebellar cortex is produced by neurons and localizes in the perineural net  <cit> .
hapln <dig> also localizes at the nodes of ranvier in the myelinated regions of the developing central nervous system  <cit> , where it plays a role in the formation of the cation diffusion barrier, important for the conduction velocity  <cit> .
the processes of neuronal differentiation and formation of the myelin sheet are related, which may explain why the genes interact.
several of the genes appearing in other interacting rules have been previously individually associated with aging.
the methylation of nptx <dig> was found to correlate with the chronological age, with older individuals having enhanced methylation  <cit> .
the gene esrp <dig>  appearing in two rules for the ‘age 28plus’ class, controls adult-specific splicing program in mouse hepatocytes  <cit> .
the gene s100a <dig>  which encodes the s <dig> calcium binding protein a <dig> and is involved in the regulation of neuritogenesis and neuronal survival  <cit> , was previously identified to be regulated in a telomerase-dependent way  <cit> .
CONCLUSIONS
we applied machine learning techniques to identify genes that contain cpg sites that change in methylation levels or patterns at particular boundaries over life span.
our approach does not aim at developing a classifier of age itself - there are other methods that can predict age using the methylation status  - but rather it introduces a method to explore which combinations of cpg sites and associated genes contribute to changing patterns at the given boundaries and methylation levels as the age change.
not surprisingly, changes in methylation were most pronounced at birth, as has been reported earlier  <cit> , resulting in comprehensive singleton rules using our approach.
conjunctive rules, i.e.
rules that associated more than one cpg site and its methylation status with an age class, showed the combinatorial role that methylation may play in more interconnected ways.
our results reported that the patterns of methylation changes in a healthy individual’s brain were highly complex and interdependent.
while we confirmed  <dig> genes related to aging or involved in diseases, notably cancer, alzheimer’s disease, and autism, that have been reported in the original study  <cit> , we identified a number of additional genes, many of which are linked to developmental and/or nervous systems specific function.
in conclusion, we expect that future studies will adopt our, or a similar machine learning approach to test these hypotheses by utilizing a multivariate analysis to compile a network of candidate cpg sites and associated genes.
additional files
additional file 1: the union of significant cpg sites  for all age cuts, showing  <dig> if a specific site is significant for a specific age.
additional file 2: figure s <dig>  histogram of beta values after filtering for standard deviations across all samples.
figure s <dig>  histogram of the samples’ ages.
table s <dig>  top  <dig> rules classifying the ‘fetus’ class.
table s <dig>  top  <dig> rules classifying the ‘age 0to4’ class.
table s <dig>  top  <dig> rules classifying the ‘age 5to27’ class.
table s <dig>  top  <dig> rules classifying the ‘age 28plus’ class.
abbreviation
mcfsmonte carlo feature selection

 dna methylation is an important epigenetic mechanism modifying mammalian genomes. it plays a major role in several biological phenomena, such as x chromosome inactivation, imprinting, regulation of gene expression, development, cell differentiation, and the onset and progression of multiple diseases. the predominant form is cytosine methylation at cpg dinucleotides. these cpg sites are located nonrandomly in the genome, tending to occur within high density clusters of cpgs. around 70 % of all cpg dinucleotides are methylated. cpg islands constitute around 60 % of human promoters and are predominantly unmethylated, while the sites in the remaining 40 % are often hypermethylated. dna methylation patterns are known to be tissue specific, and it has been shown that one of the effects of dna methylation is to contribute to transcriptional silencing, where proteins bind directly to methylated dna and recruit co-repressor complexes, triggering the formation of repressive chromatin. there are several studies that show that changes in dna methylation are not confined to early development, but rather occur over the entire life span of an organism, resulting in distinct age-related methylation profiles. while this global ‘epigenetic clock’ correlates strongly with chronological age and could thus be used to evaluate or exclude age-related factors in analyses of neurodevelopmental or neurodegenerative disorders, the specific genes or networks that depend on this clock for their regulation remain largely unknown. since the changes in methylation patterns are complex and likely involve multiple genes in a combinatorial manner, in this study our goal was to explore how a non-linear multivariate machine learning approach, capable of analyzing multiple cpg sites simultaneously, would interpret the data without any a priori hypothesis such as the direction or trajectory in which the changes in methylation occur. we thus applied a rule-based approach to a public methylation dataset profiled from the prefrontal cortex of the brain, for which we first examined changes across all age boundaries. after applying monte carlo feature selection to rank the cpg sites by significance, we identified five distinct age groups, with marked transitions between them. we then used rosetta, which implements rough sets theory, to construct rule-based models based on the identified cpg loci. methods data preprocessing the data set used in this work, numata et al., comprises dna methylation data from samples, taken from individuals ranging from fetal to 84 years old, designed to study the dependence of methylation on age and gender. genomic dna has been taken from dorsolateral prefrontal cortex. illumina’s infinium humanmethylation beadchip was used to profile the dna methylation level at cpg dinucleotides. we removed sites from the dataset if they fulfilled one or more of the following conditions: cpg sites fall on chromosome x; potentially nonspecific or polymorphic probes present on infinium humanmethylation beadchip; or cpg sites with standard deviation of beta values <  to remove uninformative sites. beta values, which were measured from a population of cells and are therefore reported as average on a scale from to were discretized into: unmethylated if the chip reports a beta value of or lower; methylated if the beta value is or higher; and intermediate if the beta value is between and. discretizing the beta values was motivated by bibikova, le, barnes et al., who divided the beta values into the three groups “methylated”, “hemimethylated”, and “unmethylated”, proposing the threshold values and based on the overall distribution of beta values. decision tables and selecting significant cpg sites we constructed decision tables as follows: each row represents a sample with the values of the characterized features of that sample in the columns. here, features are the selected cpg sites with their methylation levels as measured by the chip. the last column holds the decision class the sample belongs to.table a fragment of a decision table the rows show the samples and the columns show the value for each property. the last column contains the decision to predict we constructed separate binary decision tables by iteratively dividing the samples into two groups: younger than a given age, or older than or equal to a given age. since there were very few samples older than we stopped the cuts at the age of in order to find the cpg sites that significantly contribute to classifying the samples to the age classes, we applied monte carlo feature selection to compute a normalized relative importance score for each feature. mcfs compensates for any imbalanced number of objects in each class. constructing classifiers figure  illustrates a schematic overview of the method. the significant cpg sites for each age were extracted from the relevant decision table. since the number of samples in different age classes was different, we ran a 100-fold under-sampling to avoid the bias of classification towards classes with more samples. in under-sampling, a new decision table is created, consisting of the samples of the smallest group, plus a randomly selected subset of samples from the other groups. this process was repeated times.fig. summary of the analytical processes we then created rule-based models using rosetta for each under-sampled decision table, and combined the results into one model for each age class. rosseta implements rough set modeling, the models of which are output in the form of human-readable if-then rules, which describe the relation between the cpg sites and the decision. the rules are either conjunctive, consisting of a conjunction of conditions, such as: if cg26158194=methylated and cg12078929=intermediate then ‘olderthan50’ accuracy=, support=21 or singleton, having only one condition, such as: if cg10608341=unmethylated then ‘youngerthan0’ accuracy = support = 30 each rule can be read as: if the condition in the if part is satisfied for a sample, the model predicts it to be a member of the decision class. the reported accuracy shows the ratio of correctly classified objects to all classified objects to a specific class, based on the model confusion matrix. the support value shows the number of samples that satisfy the condition. rosetta outputs a list of rules that are independent of each other, i.e. if multiple sites, or combinations thereof, act as “predictors”, it reports all of them. this, in turn, allows for interpreting the set of cpg sites used in the rules as being subject to changes in methylation at each specific age. to avoid over-fitting of the data, we computed the statistical significance for each rule using a hyper-geometric distribution and bonferroni-correction, discarding rules with p > . we used the web based tool ciruvis to identify interactions among rule conditions, i.e. rules that use the same cpg sites, ranked by strength - the sum of accuracies multiplied by support for all the rules in which it appears. discretization into age groups we computed the jaccard distance between all two-class decision tables based on the number of overlaps between the significant features obtained for each individual two-class decision table, i.e. given two significant features for age sets sfai and sfaj for decision classes i and j, the distance is computed as: distance = – /) annotation of sites and rules we annotated the cpg sites using annovar, allowing for identifying the genomic region in which a cpg site was located, using the tags exonic, intronic, utr utr intergenic, splicing, and upstream. functional annotation for the genes and the biological processes they are involved in was obtained from genecards.  informative cpg sites in the numata et al. dataset, methylation levels measured by the illumina infinium humanmethylation beadchip are reported as beta values, i.e. the methylation averaged over the cell population that was sampled, at cpg dinucleotide sites. of these, we first removed sites located on chromosome x to avoid sites in pseudo-autosomal regions on chromosome y in male participants. we next removed polymorphic loci, and loci located on non-uniquely mapped probes. for the remaining sites, we computed the standard deviation of beta values, and discarded sites that showed little variation in methylation throughout age, by applying a cutoff of, resulting in around cpg sites.fig. histogram of standard deviation of beta values for all the cpg sites after discarding the sites located on the x chromosome or located on nonspecific or polymorphic probes discerning cpg sites over age for any given age, we next binned the samples into two categories, either below or above this age, and repeated the process times for ages to we then applied monte carlo feature selection to the respective two-class decision table to determine the set of cpg sites that significantly contribute to the classification. the number of such sites decreased as age increases, indicating that fewer sites are able to discern between samples at higher age and implying that changes in methylation are generally more pronounced at early ages and decrease with age.fig. the number of significant cpg sites plotted for all two-class decision tables we next examined the union of all significant cpg sites, in total, for functional categories. in this file, the coding is “0” for the sites not to be significant to discerning the age category, and “1” for the sites that are significant, marking genes also found by numata et al. in bold. genes overlapping with these sites are involved in various developmental, proliferation and differentiation processes, but also in brain or neuron specific functions and diseases, as expected. a total of cpg sites only contributed to the classification of age, consistent with the trend that was reported in the original study. the specific genes associated with these include artn, kmo, kcna snrk, slc1a haao, fev, chrd, spon hesx otof, elavl and sult1c which are involved in brain and/or neuron specific processes, as well as slc1a which is implicated in schizophrenia and other mental illnesses. the site annotated to a 5′utr region of this gene showed an increase of methylation level from intermediate to methylated as age increases. a site that changed from unmethylated in fetus to intermediate in adulthood was located upstream of fev, a gene implicated in the pathophysiology of psychiatric disorders such as depression, anxiety and eating disorders. another site showing the same pattern is located in an intron of spon which codes for a protein promoting adhesion and outgrowth of hippocampal embryonic neurons, as well as a site in the 5′ untranslated region of hesx which codes for a transcriptional repressor protein in the developing forebrain. we further identified genes involved in the regulation of processes such as cell migration, cell differentiation and cell progression, namely s100a arhgap and lims there were sites exclusive to classifying age which were associated with the genes msx myl mpg, actn nags, dydc1-dydc scn4b, hydin, tst, lft, ctsz, emp and net there were fewer sites involved in classifying older ages, located in genes such as ank hoxa tmem atp8a tex and tinagl several cpg sites were involved in classification of a wide range of ages, some of which covered well-defined intervals. for example, the sites associated with nptx, chrnb, rab and esrp were reported as significant for all ages between and classification into age intervals using the jaccard distance as a measure for the similarity between the calculated significant cpg sites for each age above we computed a full distance matrix and applied hierarchical clustering in r. there are three distinct groups comprising age to age to and age to with the latter exhibiting sub-groups within their respective clades. the overall topology also reflects the trend observed earlier, in that age groups become broader with increasing age, following this clustering, we constructed a four-class decision table based the classes: ‘fetus’, ‘age 0–4’, ‘age 5–27’ and ‘age28plus’. mcfs identified significant cpg sites on this set, on which we trained a classifier using rosetta. the resulting model had an accuracy mean of 90 %, as measured by 10-fold cross-validation.fig. hierarchical clustering of the significant cpg sites obtained from running mcfs on all two-class decision tables. the distance between every two significant cpg sets is based on the number of overlapping sites analysis of the rule models the model consisted of a set of rules that classify samples to a decision class according to the values of the significant features. for further analyzing these rules, we first filtered by significance, accuracy, and support. the top five most significant rules for each of the age classes are listed in additional file we tested the sites in each rule against a multivariate linear regression model, and highlighted the rules not significant in the regression in red. as expected, all singleton rules show significance in linear regression, whereas more than half of the conjunctive rules do not. table  lists the confusion matrix, indicating that the classification power for fetus and age 0– is higher than the other groups. the fetus class comprised all of the singleton rules, which is consistent with marked methylation level changes at birth, where single cpg sites are indicative for this transition. by contrast, the majority of rules classifying the older age groups comprise were conjunctive, i.e. containing a combination of sites.table confusion matrix for the rule-based classifier with four decision classes. the number of samples in each class is shown in parenthesis fig. plot of the beta values of all samples for cg annotated to the upstream of fev that classifies fetus samples. the red line shows the margin between unmethylated and intermediate beta values. if cg24178740 = unmethylated then ‘fetus’. support is and accuracy equals to as all the objects classified as fetus turn out to be fetus fig. plot of the beta values for two cpg sites appearing in the top significant rule for classifying age51plus: if cg27554782 = intermediate and cg21053323 = intermediate then ‘age51plus’. accuracy equals to and support equals to for this rule. the red lines show the margin between unmethylated and intermediate beta values while the classifier for age 0– contained conjunctive rules with perfect classification power on the data set, the rule accuracy dropped for both the age 5– and age 28+ classes, while the number of conjuncts increased, indicating that the changes in methylation become less dramatic and have thus less power to discern the samples by learning from their methylation patterns. when examining the direction in which methylation changes, we found that 81 % of the fetus rules showed a pattern of lower-to-higher methylation when transitioning into the next group. the associated cpg sites were linked to genes involved in cellular development, growth and proliferation, and neuronal related pathways and mechanisms. for example, the site, located within the tgfbr gene, whose decreased expression has been observed in various cancers, showed a methylation level increase from below in fetus to in adulthood. a similar pattern was observed for cg upstream of artn, which is a member of the glial cell-derived neurotrophic factor family of ligands, as was the site cg upstream of tp53i a gene induced by the tumor suppressor p which increased in methylation levels to in adulthood. sites showing the reverse pattern were associated with plek, pik3c2b, rd kcna acap and gpr37l genes.table list of significant singleton rules classifying fetus samples all of these rules are perfect rules with accuracy of and support of the cg site was annotated to the upstream region of nptx this gene encodes neuronal pentraxin ii, which is involved in neuritic outgrowth, synapse remodeling and the aggregation of neurotransmitter receptors at synapses. the nptx gene is reported to be hypermethylated in tumors, including brain tumors and in this data showed an increasing pattern from towards from fetus towards age 5– class and up to in the age 28+ class. the site annotated to fbxo also appeared in the rules with a similar pattern. fbxo is related to alzheimer disease, as it regulates app processing. rules interactions we used ciruvis to investigate the combinations of the features in the rules. figure  depicts the outcome for each of the decision classes. two features are connected inside the circle if they co-occur in multiple rules. the connections are shown as edges between the nodes. the width and color of the edges reflect the connection score. table  lists the two strongest interactions for all classes.fig. top strong interactions among the cpg sites for the rules, generated by ciruvis, for fetus, age 0–, age 5– and age 28+ classes. the outer ring lists the cpg sites and the inner ring shows the sites connected to each of them by an edge. the thicker and darker the edge, the stronger the interaction some of these interactions indicate coordinated age-related changes in methylation of genes involved in the same biological function or process. for example, gene tgfbr encodes tgf beta-receptor iii that serves as a co-receptor for other types of tgf beta-receptors. conversely, ectodomain shedding of tgfbr produces soluble tgfbr which inhibits tgf beta signaling. in either case, changes in the expression of tgfbr are likely to affect the signaling through the tgf beta pathway. the other gene listed as interacting with tgfbr namely cdkn2b, encodes cyclin-dependent kinase inhibitor 2b, which is a potent inhibitor of the cell cycle. in epithelial cells, cdkn2b is known to be induced by tgf beta. the link between tgfbr cdkn and the cell cycle is likely relevant to the aging brain, as tgf beta stimulates proliferation of microglia, brain resident macrophages, which play important role during aging. another strong interaction lists gene tp together with hapln gene tp encodes the tumor protein a member of the p family of transcription factors, with a role in neuronal differentiation and hippocampal development. its deficiency results in impaired self-renewal and premature differentiation of mouse neuronal progenitors. tp is also a major survival factor for postmitotic neurons. gene hapln encodes brain-specific hyaluronyan and proteoglycan link protein this protein in the cerebellar cortex is produced by neurons and localizes in the perineural net. hapln also localizes at the nodes of ranvier in the myelinated regions of the developing central nervous system, where it plays a role in the formation of the cation diffusion barrier, important for the conduction velocity. the processes of neuronal differentiation and formation of the myelin sheet are related, which may explain why the genes interact. several of the genes appearing in other interacting rules have been previously individually associated with aging. the methylation of nptx was found to correlate with the chronological age, with older individuals having enhanced methylation. the gene esrp appearing in two rules for the ‘age 28plus’ class, controls adult-specific splicing program in mouse hepatocytes. the gene s100a which encodes the s calcium binding protein a and is involved in the regulation of neuritogenesis and neuronal survival, was previously identified to be regulated in a telomerase-dependent way.  we applied machine learning techniques to identify genes that contain cpg sites that change in methylation levels or patterns at particular boundaries over life span. our approach does not aim at developing a classifier of age itself - there are other methods that can predict age using the methylation status - but rather it introduces a method to explore which combinations of cpg sites and associated genes contribute to changing patterns at the given boundaries and methylation levels as the age change. not surprisingly, changes in methylation were most pronounced at birth, as has been reported earlier, resulting in comprehensive singleton rules using our approach. conjunctive rules, i.e. rules that associated more than one cpg site and its methylation status with an age class, showed the combinatorial role that methylation may play in more interconnected ways. our results reported that the patterns of methylation changes in a healthy individual’s brain were highly complex and interdependent. while we confirmed genes related to aging or involved in diseases, notably cancer, alzheimer’s disease, and autism, that have been reported in the original study, we identified a number of additional genes, many of which are linked to developmental and/or nervous systems specific function. in conclusion, we expect that future studies will adopt our, or a similar machine learning approach to test these hypotheses by utilizing a multivariate analysis to compile a network of candidate cpg sites and associated genes. additional files additional file 1: the union of significant cpg sites for all age cuts, showing if a specific site is significant for a specific age. additional file 2: figure s histogram of beta values after filtering for standard deviations across all samples. figure s histogram of the samples’ ages. table s top rules classifying the ‘fetus’ class. table s top rules classifying the ‘age 0to4’ class. table s top rules classifying the ‘age 5to27’ class. table s top rules classifying the ‘age 28plus’ class. abbreviation mcfsmonte carlo feature selection 
our rule-based classifier reports combinations of cpg sites, together with changes in their methylation status in the form of easy-to-read if-then rules, which allows for identification of the genes associated with the underlying sites. monte carlo feature selection produced a list of ranked and statistically significant cpg sites, while rule-based models allowed for identifying particular methylation changes in these sites. dna methylation plays a key role in developmental processes, which is reflected in changing methylation patterns at specific cpg sites over the lifetime of an individual. we utilized machine learning and statistical methods to discretize decision class values to get a general pattern of methylation changes over the lifespan. we applied a multivariate approach to identify combinations of cpg sites that undergo modifications when transitioning between developmental stages. the cpg sites present in the significant rules were annotated to genes involved in brain formation, general development, as well as genes linked to cancer and alzheimer’s disease. dna methylationagingrule-based classificationfeature selectionsvenska forskningsrådet formas polish national science centreissue-copyright-statement© the author 2016 the underlying mechanisms are complex and possibly affect multiple genes or entire pathways.
 dna methylation plays a key role in developmental processes, which is reflected in changing methylation patterns at specific cpg sites over the lifetime of an individual. the underlying mechanisms are complex and possibly affect multiple genes or entire pathways.  we applied a multivariate approach to identify combinations of cpg sites that undergo modifications when transitioning between developmental stages. monte carlo feature selection produced a list of ranked and statistically significant cpg sites, while rule-based models allowed for identifying particular methylation changes in these sites. our rule-based classifier reports combinations of cpg sites, together with changes in their methylation status in the form of easy-to-read if-then rules, which allows for identification of the genes associated with the underlying sites.  we utilized machine learning and statistical methods to discretize decision class values to get a general pattern of methylation changes over the lifespan. the cpg sites present in the significant rules were annotated to genes involved in brain formation, general development, as well as genes linked to cancer and alzheimer’s disease. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords dna methylationagingrule-based classificationfeature selectionsvenska forskningsrådet formas polish national science centreissue-copyright-statement© the author 2016 