Transcription Factor Activity Inference: Does it really work?
Creators: name: Ma, Cynthia Z.
name: Brent, Michael R.
Publisher: name: F1000 Research Limited
Year: 2019

description: <i>Summary</i><br>Cells process and respond to information from their environments, in part, by changing the activity levels of transcription factors – the degree to which each transcription factor (TF) exerts its regulatory potential on its target genes. TF activity (TFA) has many possible molecular implementations and is therefore difficult to measure directly, which both motivates the development of inference methods, and bottlenecks the validation of any inferred TFA values. In this poster, we present an evaluation framework that allows us to both infer and validate the TFA values of a simple gene regulation model using assumptions about the activity of TFs in direct perturbation samples: The inferred activity of a TF in its knock-out or over-expression perturbation sample should be less or greater, respectively, than its inferred activity in the wild-type (WT) sample; The inferred activity of a TF in its knock-out or over-expression perturbation sample should be the smallest or greatest, respectively, of all TFs whose values are inferred for that sample; The inferred activity of a TF whose activity is known to be regulated by the perturbed gene of a sample should be effected more than the activities of other TFs whose values are inferred for that sample.Lastly, we add one additional validation metric that does not depend on perturbation data: The inferred activity pattern of a TF should be positively correlated with the measured mRNA level of the gene that encodes that TF, more often than negatively correlated. Using these four metrics, we show that a gene regulation model optimized on perturbation data and a network map based on ChIP or DE support can score significantly higher than a random chance model for all four metrics (p&lt;0.001), demonstrating conclusively that TFA inference does work, as well as revealing the performance gap that a better inference method should strive to fill in future research.<br><br><i>To read this poster</i><br>The flow diagram labeled "Evaluation framework" illustrates how we manage to fairly use perturbation data for both inferring and validating TFA values. In the top third, a TF-target gene regulation network and perturbation data set 1 is used to optimize a gene-regulation model. The network is built as described in the section "Building qualitative networks," the gene-regulation model used is described in the section "Bi-linear model," and the optimization process is described in the section "Optimization." In the middle third, the optimized network is used to fit new TFA values to perturbation data set 2, this time without knowledge of what was perturbed or how. In the last third, the second set of TFA values are validated using metrics defined by the assumptions listed above:<br>Correlation -&gt; Fraction Positive: the fraction of TFs whose inferred activity levels were positively correlated with their mRNA levels (abbreviated to Pos. Corr. in figures)<br>Perturbed -&gt; Median Rank Percentile: the median rank of a TF in its perturbation sample, across all perturbation samples, converted to a percentile so that rank 1 is 100% while rank 2 out of fifty TFs is 98% (abbreviated to Perturb. Rank in figures)<br>Perturbed -&gt; Fraction Correct Direction: the fraction of TFs whose inferred activity levels in their perturbation sample were correctly inferred less or greater than in the WT sample (abbreviated to Directed in figures)<br>Literature -&gt; Median Rank Percentile: the median rank of a TF in a sample where its known activity regulator is perturbed, across all tested regulator-TF relationships, converted to a percentile so that rank 1 is 100% while rank 2 out of fifty TFs is 98% (abbreviated to Lit. Rank in figures)<br><br>For the figures in the section labeled "Performance on validation tasks," asterisks show the significance of results, where * indicates a p-value &lt;= 0.01, ** indicates a p-value &lt;= 0.001, and *** indicates a p-value &lt;= 0.0001.<br>ChIP, DE, and PWM refers to the network map having been built using support from chromatin immunoprecipitation, differential expression, and positive weight matrices respectively.<br>-CC and -PC refers to constraints being based on correlation between TFs and target genes or TF perturbation induced change in gene expression respectively.<br>In the first figure, it's shown the necessity of optimizing the network values (Control Strength values) along with the TFA values, rather than just using the constraints and structure represented by the signed binary network, or randomly generated Control Strength values.<br>The Blocks in the second figure are the edges that are used for network mapping, after breaking all the ranked edges into blocks of 2000; e.g. Block 1 includes the top 2000 rank edges, Block 2 includes the edges ranked top 2001-4000, and Block 4 includes the edges ranked top 6001-8000. The figure shows that as support for network edges decrease, so does TFA inference performance.<br>For the last figure, it's shown that the best performance across the four metrics depends on constraints defined by TF perturbation data, and network structure defined by ChIP and DE data. In other words, perturbation data is both necessary and sufficient for good TFA inference.