We investigated the intracellular metabolic mechanisms driving oxLDL-induced trained immunity in human primary monocytes and observed concomitant upregulation of glycolytic activity and oxygen consumption. In two separate cohorts of healthy volunteers, we assessed the impact of genetic variation in glycolytic genes on the training capacity of monocytes and found that variants mapped to glycolytic enzymes PFKFB3 and PFKP influenced trained immunity by oxLDL. Subsequent functional validation with inhibitors of glycolytic metabolism revealed dose-dependent inhibition of trained immunity in vitro. Furthermore, in vivo administration of the glucose metabolism modulator metformin abrogated the ability for human monocytes to mount a trained response to oxLDL. These findings underscore the importance of cellular metabolism for oxLDL-induced trained immunity and highlight potential immunomodulatory strategies for clinical management of atherosclerosis.Stimulation of monocytes with microbial and non-microbial products, including oxidized low-density lipoprotein (oxLDL), induces a protracted pro-inflammatory, atherogenic phenotype sustained by metabolic and epigenetic reprogramming via a process called Brief stimulation of monocytes to oxLDL induces a prolonged inflammatory phenotype.This is due to upregulation of glycolytic metabolism.Genetic variation in glycolytic genes modulates oxLDL-induced trained immunity.Pharmacological inhibition of glycolysis prevents trained immunity.The online version of this article (10.1007/s00109-020-01915-w) contains supplementary material, which is available to authorized users. We receimmunity , 3. Impoimmunity –6. Cellsimmunity . Therefoimmunity . To thisimmunity , 9. Indeimmunity , 11.Trained immunity induced by β-glucan or BCG is associated with profound intracellular metabolic reprogramming, characterized by increased glycolytic metabolism and intracellular accumulation of fumarate and mevalonate , 12–14. At the level of gene regulation, trained immunity is characterized by epigenetic changes that modulate transcriptional programs. Studies of cells trained with β-glucan  and BCG The current study is aimed at unraveling the role of metabolic reprogramming in oxLDL-induced trained immunity.Escherichia coli lipopolysaccharide , Pam3Cys , 3-(3-pyridinyl)-1-(4-pyridinyl)-2-propen-1-one , and 2-deoxy-d-glucose . Low-density lipoprotein was isolated from pooled human serum by ultracentrifugation and oxidized by incubating with 20 μmol CuSO4/L for 16 h at 37 °C followed by dialysis, as previously described [Buffy coats from healthy donors were obtained after written informed consent . Human peripheral blood mononuclear cells (PBMCs) were isolated by density-gradient centrifugation over Ficoll-Paque . Percoll isolation of monocytes was performed as previously described as yielding a level of T cell contamination, measured by fluorescence-activated cell sorting of only 5% , 17. Pur2. For pharmacological inhibition experiments, cells were co-incubated with inhibitors  for the 24 h of oxLDL stimulation. For glucose experiments, cells were incubated with oxLDL (10 μg/mL) in culture medium supplemented with 5 mM glucose (+ 20 mM mannitol) or 25 mM glucose for 24 h, washed with warm PBS, and incubated with RPMI supplemented with 6 mM glucose and 10% pooled human serum  at 37 °C, 5% CO2. Following 5 days in culture, cells were restimulated with medium alone, 10 ng/mL LPS.Adherent monocytes were trained as described previously . BrieflyCytokine production in supernatants after 24 h or 7 days was determined using commercial enzyme-linked immunosorbent assay kits for TNF-α and IL-6  according to the instructions of the manufacturers.18s was used as a housekeeping gene. RT-PCR primers are listed in Table S1.Total RNA was isolated from macrophages using TRIzol reagent according to the manufacturer’s instructions. 0.5–1 μg of total RNA was used to synthesize cDNA with the SuperScript III First-Strand Synthesis System (Thermo Fisher Scientific) according to the manufacturer’s protocol. Quantitative RT-PCR was performed using an Applied Biosciences StepOnePLUS qRT-PCR machine using SYBR Green (Invitrogen). All reactions were performed for at least 6 biological replicates and the values expressed as log2 fold increase in mRNA levels relative to those in non-trained cells. Trained monocytes on day 6 were cross-linked in methanol-free 1% formaldehyde, followed by sonication and immunoprecipitation using antibodies against H3K4me3 . Immunoprecipitated chromatin was processed further for qRT-PCR analysis using the MinElute DNA Purification Kit (Qiagen). Primers used in the reaction are listed in Table 7 monocytes were trained with oxLDL (10 μg/mL) in 10-cm Petri dishes (Greiner) in 10 mL medium volumes for 24 h, washed with warm PBS, and incubated in normal culture medium at 37 °C, 5% CO2. Following 5 days in culture, cells were detached with Versene solution (Thermo Fisher Scientific) and 1 × 105 cells were plated in quintuplicate to overnight-calibrated cartridges in assay medium  and incubated for 1 h in a non-CO2-corrected incubator at 37 °C. Trained and untrained macrophages for each respective donor were included in the same assay. Oxygen consumption rate (OCR) and extracellular acidification rate (ECAR) were measured using a Cell Mito Stress Kit (for OCR) or a glycolysis stress test (for ECAR) kit in an XFp Analyzer (Seahorse Bioscience), with final concentrations of 1 μmol/L oligomycin, 1 μmol/L FCCP, and 0.5 μmol/L rotenone/antimycin A.Approximately 1 × 10www.humanfunctionalgenomics.org). Genotype information on approximately 8 million single-nucleotide polymorphisms (SNPs) was obtained using Illumina HumanOmniExpressExome SNP chip upon imputation. Only SNPs with a minor allele frequency of ≥ 5% that passed standard quality filters were included in the analysis. Raw cytokine levels were log-transformed and the ratio between trained and non-trained cytokine levels was used to quantify the trained immunity response. They were subsequently mapped to genotype data using a linear regression model with age and gender as covariates [We conducted in vitro oxLDL training of adherent PBMCs from 119 healthy individuals of Western European ancestry from the 200 Functional Genomics cohort (2011/399) of the Human Functional Genomics Project .We also conducted in vitro oxLDL training of adherent PBMCs in a second cohort of 243 healthy individuals of Western European ancestry from the 300BCG cohort (NL58553.091.16). DNA samples of the individuals from the second cohort were genotyped using the commercially available SNP chip, Infinium Global Screening Array MD v1.0 from Illumina. Genotype information on approximately 4 million SNPs was obtained upon imputation . Inclusion of volunteers and experiments were conducted according to the principles expressed in the Declaration of Helsinki. All volunteers gave written informed consent before any material was taken.In this prospective study, 11 healthy non-obese volunteers received increasing doses of metformin for a total of 6 days (starting at 500 mg once per day and ending with 1000 mg twice per day). Baseline characteristics of the participants are described in Table p value < 0.05 (*) was considered statistically significant, (**) p < 0.01. Data represent mean ± SEM.Ex vivo and in vitro monocyte experiments were analyzed using a Wilcoxon signed-rank test. R-package Matrix eQTL was used for cytokine QTL mapping. A To investigate the metabolic phenotype of monocytes trained with oxLDL, we incubated human primary monocytes with culture medium or oxLDL (10 μg/mL) for 24 h. On day 6, the cells were restimulated with culture medium alone or the Toll-like receptor 4 ligand lipopolysaccharide  for 24 h, after which pro-inflammatory cytokine production was measured Fig. . Tumor nPFKP, PKM1, and PKM2 in cells incubated with RPMI. Statistically significant differences were observed for HK2 and PFKFB3 in untrained macrophages stimulated with LPS. Training with oxLDL exacerbated this effect, particularly for PFKFB3 mRNA expression. In contrast, the expression of genes encoding pyruvate kinase enzymes PKM1 and PKM2 was not significantly altered by oxLDL training  Fig. , signifying Fig. .. Furthermore, we observed considerable inter-individual variation in cytokine production by oxLDL-trained cells . To investigate the sources of this variation, we explored the potential influence of factors known to affect cytokine production and observed no effect of age  for 24 h. We confirmed that the oxLDL-dependent augmented production of TNF-α and IL-6 previously seen in enriched monocyte fractions was also detectable in PBMCs Fig. . Furtherlls Fig. . To inveage Fig.  or sex  QTLs were not observed. However, to increase sensitivity, we studied all SNPs with a p value < 9.99 × 10−3 Using this approach, we identified several SNPs suggestively associated with adaptive changes in cytokine production mapped within 250 kB of genes encoding key glycolytic enzymes. Specifically, genetic variation in genes encoding the inducible PFK-2/FBPase isozyme 6-phosphofructo-2-kinase/fructose-2,6-biphosphatase 3 (PFKFB3) and phosphofructokinase (PFKP) were associated with the potentiation of TNF-α and IL-6 production upon training with oxLDL  which is located within an enhancer region approximately 100 kB downstream of PFKP (GeneHancer ID: GH10J003171). For IL-6 production, the SNP most strongly associated with oxLDL training was rs4747882    and rs55643411 , whereas SNPs strongly associated with IL-6 production included rs10762282  and rs55643411   Fig. . We vali−3) Fig. . Like th01) Fig. . TogetheTo determine the physiological significance of changes in glycolytic metabolism, we used two distinct approaches to pharmacologically target glycolysis. First, we inhibited PFKFB3, a crucial rate-limiting enzyme in glycolysis, which is upregulated in oxLDL-trained cells . Co-incuPFKFB3 and HK2 induced by training with oxLDL, which was also attenuated when glycolysis was inhibited by 2-DG. On the other hand, we did not observe H3K4me3 enrichment at the PFKP promoter, which is in line with the lack of transcriptional upregulation following LPS exposure  risk is severely elevated in individuals with diabetes . MoreoveMetformin is the antihyperglycemic drug of first choice in patients with type 2 diabetes mellitus. While the mechanism of action is complex, metformin is known to activate AMP-activated protein kinase and subsequently inhibit mechanistic target of rapamycin (mTOR) . In addiLdlr−/− mice and likely represents an important novel mechanism in atherogenesis [The ability of human innate immune cells to build a de facto immunological memory of infectious challenges has been recently described . In paraogenesis . We now HK2) and PFKFB3 is significantly elevated in monocytes isolated from patients with symptomatic atherosclerosis [Pfkfb3 in high fat-fed ApoE−/− mice correlated with a significant reduction in aortic tissue levels of TNF-α and CCL2 [The importance of changes in cellular glucose metabolism was previously indicated for the development of atherosclerosis . In atheclerosis . Furtherand CCL2 . This asand CCL2 .HK2 and PFKP modulates the induction of trained immunity by BCG [Here, we describe the critical importance of glucose metabolism for monocytes to build an immunological memory after oxLDL stimulation, underwriting their capacity to mount a subsequently heightened response to TLR stimulation. Our results closely mirror aspects of the metabolic reprogramming induced by microbial stimulators of trained immunity. Indeed, genetic variation in y by BCG . FurtherBy supplying the energy required for different states of activation, metabolic pathways distinguish and support the spectrum of macrophage phenotypes. Though energetically less efficient than OXPHOS, a potential explanation for the dependency on glycolysis is that this metabolic pathway can be rapidly amplified to meet the ATP requirements of trained cells, and that the ATP producing capacity of the TCA cycle is limited by anaplerotic repurposing of TCA cycle intermediates, as described for β-glucan-induced training . Indeed,Our observation that pharmacological blockade of glycolysis prevents oxLDL-mediated trained immunity suggests that this pathway represents a potential therapeutic target to prevent ASCVD. Similar to findings reported for β-glucan-induced trained immunity , inhibitAccelerated atherosclerosis is the principal cause of mortality in patients with diabetes . Our obsIn conclusion, we show distinguishable changes in the glucose metabolism of human primary monocytes mediated by brief exposure to a low concentration of oxLDL. Our cohort analyses revealed the importance of genetic variation in glycolytic regulators for the induction of trained immunity, and by targeting this metabolic pathway, we demonstrate the critical importance of glycolysis for the induction of a pro-inflammatory monocyte phenotype in oxLDL-trained macrophages. We propose that strategies interfering with glucose utilization specifically in the context of trained immunity may represent novel approaches to the treatment of vascular inflammation and atherosclerosis. However, further studies are necessary to strengthen the connection between our findings and human disease. For example, the use of recombinant high-density lipoprotein nanoparticles to deliver statins directly to macrophages in atherosclerotic lesions  could beESM 1(PDF 53 kb).ESM 2(PDF 50 kb).


A systematic and reproducible “workflow”—the process that moves a scientific investigation from raw data to coherent research question to insightful contribution—should be a fundamental part of academic data-intensive research practice. In this paper, we elaborate basic principles of a reproducible data analysis workflow by defining 3 phases: the Explore, Refine, and Produce Phases. Each phase is roughly centered around the audience to whom research decisions, methodologies, and results are being immediately communicated. Importantly, each phase can also give rise to a number of research products beyond traditional academic publications. Where relevant, we draw analogies between design principles and established practice in software development. The guidance provided here is not intended to be a strict rulebook; rather, the suggestions for practices and tools to advance reproducible, sound data-intensive analysis may furnish support for both students new to research and current researchers who are new to data-intensive work. The importance and effective development of a workflow should, in turn, be a cornerstone of the data science education designed to prepare researchers across disciplinary specializations.Both traditional science fields and the humanities are becoming increasingly data driven and computational. Researchers who may not identify as data scientists are working with large and complex data on a regular basis. A systematic and reproducible computational tools, software, and programming languages. In scientific fields, education and training includes a review of domain-specific methods and tools, but generally omits guidance on the coding practices relevant to developing new analysis software—a skill of growing relevance in data-intensive scientific fields [reproducible science (providing the necessary data and code to recreate the results) and effective career building, researchers should be primed to regularly generate outputs over the course of their workflow.Data science education tends to review foundational statistical analysis methods  and furnc fields . Meanwhidata-intensive research project, demonstrating how both design principles and software development methods can motivate the creation and standardization of practices for reproducible data and code. The implementation of such practices generates research products that can be effectively communicated, in addition to constituting a scientific contribution. Here, “data-intensive” research is used interchangeably with “data science” in a recognition of the breadth of domain applications that draw upon computational analysis methods and workflows. .Assertion: An expression that is expected to be true at a particular point in the code.Computational tool: May include libraries, packages, collections of functions, and/or data structures that have been consciously designed to facilitate the development and pursuit of data-intensive questions (synonymous term: software tool).Continuous integration: Automatic tests that updated code.Gut check: Also “data gut check.” Quick, broad, and shallow testing [ testing  before a testing , consistData-intensive research: Research that is centrally based on the analysis of data and its structural or statistical properties. May include but is not limited to research that hinges on large volumes of data or a wide variety of data types requiring computational skills to approach such research (synonymous term: data science research). “Data science” as a stand-alone term may also refer more broadly to the use of computational tools and statistical methods to gain insights from digitized information.Data structure: A format for storing data values and definition of operations that can be applied to data of a particular type.Defensive programming: Strategies to guard against failures or bugs in code; this includes the use of tests and assertions.Design thinking: The iterative process of defining a problem then identifying and prototyping potential solutions to that problem, with an emphasis on solutions that are empathetic to the particular needs of the target user.Docstring: A code comment for a particular line of code that describes what a function does, as opposed to how the function performs that operation.DOI: A digital object identifier or DOI is a unique handle, standardized by the International Organization for Standardization (ISO), that can be assigned to different types of information objects.Extensibility: The flexibility to be extended or repurposed in a new scenario.Function: A piece of more abstracted code that can be reused to perform the same operation on different inputs of the same type and has a standardized output [d output –52.Getter function: Another term for an accessor function.Integrated Development Environment (IDE): A software application that facilitates software development and minimally consists of a source code editor, build automation tools, and a debugger.Modularity: An ability to separate different functionality into stand-alone pieces.Mutator method: A function used to control changes to variables. See “setter function” and “accessor function.”Notebook: A computational or physical place to store details of a research process including decisions made.Mechanistic code: Code used to perform a task as opposed to conduct an analysis. Examples include processing functions and plotting functions.Overwrite: The process, intentional or accidental, of assigning new values to existing variables.Package manager: A system used to automate the installation and configuration of software.Pipeline: A series of programmatic processes during data analysis and data cleaning, usually linear in nature, that can be automated and usually be described in the context of inputs and outputs.Premature optimization: Focusing on details before the general scheme is decided upon.Refactoring: A change in code, such as file renaming, to make it more organized without changing the overall output or behavior.Replicable: A new study arrives at the same scientific findings as a previous study, collecting new data (with the same or different methods) and completes new analyses [analyses –55.Reproducible: Authors provide all the necessary data, and the computer codes to run the analysis again, recreating the results [ results –55.Script: A collection of code, ideally related to one particular step in the data analysis.Setter function: A type of function that controls changes to variables. It is used to directly access and alter specific values (synonymous term: mutator method).Serialization: The process of saving data structures, inputs and outputs, and experimental setups generally in a storable, shareable format. Serialized information can be reconstructed in different computer environments for the purpose of replicating or reproducing experiments.Software development: A process of writing and documenting code in pursuit of an end goal, typically focused on process over analysis.Source code editor: A program that facilitates changes to code by an author.Technical debt: The extra work you defer by pursuing an easier, yet not ideal solution, early on in the coding process.Test-driven development: Each change in code should be verified against tests to prove its functionality.Unit test: A code test for the smallest chunk of code that is actually testable.Version control: A way of managing changes to code or documentation that maintains a record of changes over time.White paper: An informative, at least semiformal document that explains a particular issue but is not peer reviewed.Workflow: The process that moves a scientific investigation from raw data to coherent research question to insightful contribution. This often involves a complex series of processes and includes a mixture of machine automation and human intervention. It is a nonlinear and iterative exercise.pipeline” in the context of software development and engineering. Pipelines are often described as a series of processes that can be programmatically defined and automated and explained in the context of inputs and outputs. However, in this paper, we offer an important distinction between pipelines and workflows: The former refers to what a computer does, for example, when a piece of software automatically runs a series of Bash or R scripts. For the purpose of this paper, a workflow describes what a researcher does to make advances on scientific questions: developing hypotheses, wrangling data, writing code, and interpreting results.Discussions of “workflow” in data science can take on many different meanings depending on the context. For example, the term “workflow” often gets conflated with the term “replicable  or academia (research papers). Rather, the workflow that a researcher defines and iterates over the course of a data science project can lead to intellectual contributions as varied as novel data sets, new methodological approaches, or teaching materials in addition to the classical tools, packages, and papers. While the workflow should be designed to serve the researcher and their collaborators, maintaining a structured approach throughout the process will inform results that are design thinking and software development. While the 3 phases described here are not intended to be a strict rulebook, we hope that the many references to additional resources—and suggestions for nontraditional research products—provide guidance and support for both students new to research and current researchers who are new to data-intensive work.In the following sections, we explain the basic principles of a constructive and productive data analysis workflow by defining 3 phases: the Explore, Refine, and Produce Phases. Each phase is roughly centered around the audience to whom research decisions, methodologies, and results are being immediately communicated. Where relevant, we draw analogies to the realm of We partition the workflow of a data-intensive research process into 3 phases: Explore, Refine, and Produce. These phases, collectively the ERP workflow, are visually described in Each phase has an immediate audience—the researcher themselves, their collaborative groups, or the public—that broadens progressively and guides priorities. Each of the 3 phases can benefit from standards that the software development community uses to streamline their code-based pipelines, as well as from principles the design community uses to generate and carry out ideas; many such practices can be adapted to help structure a data-intensive researcher’s workflow. The Explore and Refine Phases provide fodder for the concurrent Produce Phase. We hope that the potential to produce a variety of research products throughout a data-intensive research process, rather than merely at the end of a project, motivates researchers to apply the ERP workflow.Data-intensive research projects typically start with a domain-specific question or a particular data set to explore . There iTrial and error is the hallmark of the Explore Phase , rather than building new computational capabilities ourselves. For example, during this phase, a common activity includes surveying the software available for our data set or problem space and estimating its utility for the unique demands of our current analysis. Through exploration, we learn about relevant computational and analysis tools while concurrently building an understanding of our data.functions therein—that are best suited to our data and domain area, we also start putting those tools to use [A second important goal of the Explore Phase is data cleaning and developing a strategy to analyze our data. This is a dynamic process that often goes hand in hand with improving our understanding of the data. During the Explore Phase, we redesign and reformat data structures, identify important variables, remove redundancies, take note of missing information, and ponder outliers in our data set. Once we have established the software tools—the programming language, data analysis packages, and a handful of the useful s to use . In addiThe Explore Phase is often a solo endeavor; as shown in premature optimization [However, there are strategies—inspired by analogous software development principles—that can help set us up for success in meeting the standards of reproducibility  relevantmization  while wedocstrings detailing function capabilities, and vignettes providing example applications. Documentation provides both a user manual for particular tools within a project , and a reference log describing scientific research decisions and their rationale .Software engineers typically value formal documentation that is readable by software users. While the audience for our data analysis code may not be defined as a software user per se, documentation is still vital for workflow development. Documentation for data analysis workflows can come in many forms, including comments describing individual lines of code, README files orienting a reader within a code repository, descriptive commit history logs tracking the progress of code development, In the Explore Phase, we may identify with the type of programmer described by Brant and colleagues as “opportunistic” . This tyEven if the code that is used for data exploration is not developed into a software-based final research product, the exploratory process as a whole should exist as a permanent record: Future scientists should be able to rerun our analysis and work from where we left off, beginning from raw, unprocessed data. Therefore, documenting choices and decisions we make along the way is crucial to making sure we do not forget any aspect of the analysis workflow, because each choice may ultimately impact the final results. For example, if we remove some data points from our analyses, we should know which data points we removed—and our reason for removing them—and be able to communicate those choices when we start sharing our work with others. This is an important argument against ephemerally conducting our data analysis work via the command line.notebook [Instead of the command line, tools like a computational notebook  can helpnotebook . A compunotebook –14. Suchnotebook  to recorversion control system such as Git. With its issues, branches, and informative commit messages, Git is another useful way to maintain a record of our trial-and-error process and track which files are progressing toward which goals of the overall project. Using Git together with a public online hosting service such as GitHub allows us to share our work with collaborators and the public in real time, if we so choose.To go a step beyond annotated scripts or notebooks, researchers might employ a A researcher dedicated to conducting an even more thoroughly documented Explore Phase may take Ford’s advice and include notes that explicitly document our stream of consciousness . Our nottest-driven development. This is a tension between the code-based work conducted in scientific research versus software development: Tests help build confidence in analysis code and convince users that it is reliable or accurate, but tests also imply finality and take time to write that we may not be willing to allocate in the experimental Explore Phase. However, software development style tests do have useful analogs in data analysis efforts: We can think of tests, in the data analysis sense, as a way of checking whether our expectations match the reality of a piece of code’s output.As Ford  explainsImagine we are looking at a data set for the first time. What weird things can happen? The type of variable might not be what we expect . The data set could also include unexpected aspects . The amount of missing data may be larger than we thought, and this missingness could be coded in a variety of ways . Finally, the dimensions of a data frame after merging or subsetting it for data cleaning may not match our expectations. Such gaps in expectation versus reality are “silent faults” . WithoutFor these reasons, every data exploration should include quantitative and qualitative “gut checks”  that candefensive programming techniques that help guard against specific mistakes. An example of defensive programming in the Julia language is the use of assertions, such as the @assert macro to validate values or function outputs. Another option includes writing “chatty functions” [We perform these manual checks to reassure ourselves that our actions at each step of data cleaning, processing, or preliminary analysis worked as expected. However, these types of checks often rely on us as researchers visually assessing output and deciding if we agree with it. As we transition to needing to convince users beyond ourselves of the correctness of our work, we may consider employing nctions”  that sigA researcher in the Explore Phase experiments with a variety of potential data configurations, analysis tools, and research directions. Not all of these may bear fruit in the form of novel questions or promising preliminary findings. Learning how to find a balance between the breadth and depth of data exploration helps us understand when to transition to the Refine Phase of data-intensive research. Specific questions to ask ourselves as we prepare to transition between the Explore Phase and the Refine Phase can be found in This box provides guiding questions to assist readers in navigating through each workflow phase. Questions pertain to planning, organization, and accountability over the course of workflow iteration.Questions to ask in the Explore PhaseGood: Ourselves Better: Our small team who has specialized knowledge about the context of the problem.Best: Anyone with experience using similar tools to us.Who can read through our materials and understand our workflow?Good: Dead ends marked differently than relevant and working code.Better: Material connected to a handful of promising leads.Best: Material connected to a clearly defined scope.What material do we think is worth continuing into the next phase?Good: Backed up in a second location in addition to our computer.Better: Within a shared space among our team .Best: Within a version control system  that furnishes a complete timeline of actions taken.Where does the code live?Good: Noted in a separate place from our code .Better: Noted in comments throughout the code itself, with expectations informally checked.Best: Noted systematically throughout code as part of a narrative, with expectations formally checked.Why did we make particular data cleaning and analysis decisions?Questions to ask in the Refine PhaseWho is in our team?Consider career level, computational experience, and domain-specific experience.What are our teammates’ skill levels?How do we communicate methodology with our teammates’ skills in mind?What reproducibility tools can be agreed upon?Are there established standards within the team that need to be adopted or conflict with our Explore Phase workflow?How can our work be packaged into impactful research products?Can we explain the same important results across different platforms ?What is the main takeaway of our findings?How can we alert these people and make our work accessible?Who will be affected by the outcomes of our work?How can we use narrative to make this clear?Why is our work important for our domain-specific field? For broader society?Questions to ask in the Produce PhaseDo we have more than 1 audience?Who is the intended audience for our research product(s)?What is the next step in our research?Can we turn our work into more than 1 publishable product?Where do we plan to publish?Consider products throughout the entire workflow.When do we expect a research product to be ready for a broader audience?See suggestions in the Tool development guide .Why should we decide to build a software tool based on our work?Imposing structure at certain points throughout the Explore Phase can help to balance our wide search for solutions with our deep dives into particular options. In an analogy to the software development world, we can treat our exploratory code as a code release—the marker of a stable version of a piece of software. For example, we can take stock of the code we have written at set intervals, decide what aspects of the analysis conducted using it seem most promising, and focus our attention on more formally tuning those parts of the code. At this point, we can also note the presence of research “dead ends” and perhaps record where they fit into our thought process. Some trains of thought may not continue into the next phase or become a formal research product, but they can still contribute to our understanding of the problem or eliminate a potential solution from consideration. As the project matures, computational pipelines are established. These inform project workflow, and tools, such as Snakemake and Nextflow, can begin to be used to improve the flexibility and reproducibility of the project –23. As wJust as Cross  finds thInevitably, we reach a point in the Explore Phase when we have acquainted ourselves with our data set, processed and cleaned it, identified interesting research questions that might be asked using it, and found the analysis tools that we prefer to apply. Having reached this important juncture, we may also wish to expand our audience from ourselves to a team of research collaborators. It is at this point that we are ready to transition to the Refine Phase. However, we should keep in mind that new insights may bring us back to the Explore Phase: Over the lifetime of a given research project, we are likely to cycle through each workflow phase multiple times.In the Refine Phase, the extension of our target audience demands a higher standard for communicating our research decisions as well as a more formal approach to organizing our workflow and documenting and testing our code. In this section, we will discuss principles for structuring our data analysis in the Refine Phase. This phase will ultimately prepare our work for polishing into more traditional research products, including peer-reviewed academic papers.serialization of experimental setups. Finally, standards of reproducibility should be maintained throughout. Each of these aspects constitutes an important goal of the Refine Phase as we determine the most promising avenues for focusing our research workflow en route to the polished research products that will emerge from this phase and demand even higher reproducibility standards.The Refine Phase encompasses many critical aspects of a data-intensive research project. Additional data cleaning may be conducted, analysis methodologies are chosen, and the final experimental design is decided upon. Experimental design may include identifying case studies for variables of interest within our data. If applicable, it is during this phase that we determine the details of simulations. Preliminary results from the Explore Phase inform how we might improve upon or scale up prototypes in the Refine Phase. Data management is essential during this phase and can be expanded to include the All of these goals are developed in conjunction with our research team. Therefore, decisions should be documented and communicated in a way that is reproducible and constructive within that group. Just as the solitary nature of the Explore Phase can be daunting, the collaboration that may happen in the Refine Phase brings its own set of challenges as we figure out how to best work together. Our team can be defined as the people who participate in developing the research question, preparing the data set it is applied to, coding the analysis, or interpreting the results. It might also include individuals who offer feedback about the progress of our work. In the context of academia, our team usually includes our laboratory or research group. Like most other aspects of data-intensive research, our team may evolve as the project evolves. But however we define our team, its members inform how our efforts proceed during the Refine Phase: Thus, another primary goal of the Refine Phase is establishing group-based standards for the research workflow. Specific questions to ask ourselves during this phase can be found in In recent years, the conversation on standards within academic data science and scientific computing has shifted from “best” practices  to “goodThe concept of literate programming  is at thThe same strategies that promote scientific reproducibility for traditional laboratory notebooks can be applied to the computational notebook . After apackage manager, simplifying matters and painlessly raising the standards of reproducibility in a research team. The unprecedented levels of reproducibility possible in modern computational environments have produced some variance in the expectations of different research communities; it behooves the research team to investigate the community-level standards applicable to our specific domain science and chosen programming language.As scientists, we should keep a record of the tools we use to obtain our results in addition to our methodological process. In a data-intensive research workflow, this includes documenting the specific version of any software that we used, as well as its relevant dependencies and compatibility constraints. Recording this information at the top of the computational notebook that details our data science experiment allows future researchers—including ourselves and our teams—to establish the precise computational environment that was used to run the original research analysis. Our chosen programming language may supply automated approaches for doing this, such as a A notebook can include more than a deep dive into a full-fledged data science experiment. It can also involve exploring and communicating basic properties of the data, whether for purposes of training team members new to the project or for brainstorming alternative possible approaches to a piece of research. In the Exploration Phase, we have discovered characteristics of our data that we want our research team to know about, for example, outliers or unexpected distributions, and created preliminary visualizations to better understand their presence. In the Refine Phase, we may choose to improve these initial plots and reprise our data processing decisions with team members to ensure that the logic we applied still holds.Computational notebooks can live in private or public repositories to ensure accessibility and transparency among team members. A version control system such as Git continues to be broadly useful for documentation purposes in the Refine Phase, beyond acting as a storage site for computational notebooks. Especially as our team and code base grows larger, a history of commits and pull requests helps keep track of responsibilities, coding or data issues, and general workflow.technical debt in the form of duplications or overwritten variables. As our research project grows in complexity and size, or gains team members, we may want to transition to an Integrated Development Environment (IDE) or a source code editor—which interact easily with container environments like Docker and version control systems such as GitHub—to help scale our data analysis, while retaining important properties like reproducibility.Importantly however, all tools have their appropriate use cases. Researchers should not develop an overt reliance on any one tool and should learn to recognize when different tools are required. For example, computational notebooks may quickly become unwieldy for certain projects and large teams, incurring mechanistic code, especially in “big data” statistical analyses or complex dynamic simulations where optimized computation becomes a concern. Keeping the immediate audience of this workflow phase, our research team, at the forefront of our mind can help us take steps to structure both mechanistic and analysis code in a useful way.Code in data-intensive research is generally written as a means to an end, the end being a scientific result from which researchers can draw conclusions. This stands in stark contrast to the purpose of code developed by data engineers or computer scientists, which is generally written to optimize a mechanistic function for maximum efficiency. During the Refine Phase, we may find ourselves with both analysis-relevant and Unit tests and so-called accessor functions or getter and setter functions that extract parameter values from data structures or set new values are examples of mechanistic code that might be included in a data-intensive research analysis. Meanwhile, code that is designed to gain statistical insight into distributions or model scientific dynamics using mathematical equations are 2 examples of analysis code. Sometimes, the line between mechanistic code and analysis code can be a blurry one. For example, we might write a looping function to sample our data set repeatedly, and that would classify as mechanistic code. But that sampling may be designed to occur according to an algorithm such as Markov Chain Monte Carlo that is directly tied to our desire to sample from a specific probability distribution; therefore, this could be labeled analysis and mechanistic code. Keep your audience in mind and the reproducibility of your experiment when considering how to present your code.Mechanistic code, which is designed for repeated use, often employs abstractions by wrapping code into functions that apply the same action repeatedly or stringing together multiple scripts into a computational pipeline. modularity while reducing the propensity for user-induced error. However, the scripts and programming notebooks so useful to establishing a narrative and documenting work in the Refine Phase are set up to be read in a linear fashion. Embedding mechanistic functions in the midst of the research narrative obscures the utility of the notebooks in telling the research story and generally clutters up the analysis with a lot of extra code. For example, if we develop a function to eliminate the redundancy of repeatedly restructuring our data to produce a particular type of plot, we do not need to showcase that function in the middle of a computational notebook analyzing the implications of the plot that is created—the point is the research implications of the image, not the code that made the plot. Then where do we keep the data-reshaping, plot-generating code?It is common practice to wrap code that we use repeatedly into functions to increase readability and Strategies to structure the more mechanistic aspects of our analysis can be drawn from common software development practices. As our team grows or changes, we may require the same mechanistic code. For example, the same data-reshaping, plot-generating function described earlier might be pulled into multiple computational experiments that are set up in different locations, computational notebooks, scripts, or Git branches. Therefore, a useful approach would be to start collecting those mechanistic functions into their own script or file, sometimes called “helpers” or “utils,” that acts as a supplement to the various ongoing experiments, wherever they may be conducted. This separate script or file can be referenced or “called” at the beginning of the individual data analyses. Doing so allows team members to benefit from collaborative improvements to the mechanistic code without having to reinvent the wheel themselves. It also preserves the narrative properties of team members’ analysis-centric computational notebooks or scripts while maintaining transparency in basic methodologies that ensure project-wide reproducibility. The need to begin collecting mechanistic functions into files separate from analysis code is a good indicator that it may be time for the research team to supplement computational notebooks by using a code editor or IDE for further code development.continuous integration is often employed [control data sets where we know the result of a particular analysis to make sure our analysis code is functioning as we expect. Alternatively, we could also use a regression test to compare computational outputs before and after changes in the code to make sure we haven’t introduced any unanticipated behavior.Testing scientific software is not always perfectly analogous to testing typical software development projects, where automated employed . Howeveremployed . Formal Workflows in data science are rarely linear; it is often necessary for researchers to iterate between the Refine and Explore Phases . For exaIteration between the Refine and Explore Phases is a careful balance. On the one hand, we should be careful not to allow “scope creep” to expand our problem space beyond an area where we are able to develop constructive research contributions. On the other hand, if we are too rigid about decisions made over the course of our workflow and refuse to look backwards as well as forwards, we may risk cutting ourselves off from an important part of the potential solution space.Agile frameworks, to help guide the careful balancing act required to conduct research that is both comprehensive and able to be completed [Data-intensive researchers can once more look to principles within the software development community, such as ompleted ,35. How In the previous sections of this paper, we discussed how to progress from the exploration of raw data through the refinement of a research question and selection of an analytical methodology. We also described how the details of that workflow are guided by the breadth of the immediately relevant audience: ourselves in the Explore Phase and our research team in the Refine Phase. In the Produce Phase, it becomes time to make our data analysis camera ready for a much broader group, bringing our research results into a state that can be understood and built upon by others. This may translate to developing a variety of research products in addition to—or instead of—traditional academic outputs like peer-reviewed publications and typical software development products such as computational tools.The main goal of the Produce Phase is to prepare our analysis to enter the public realm as a set of products ready for external use, reflection, and improvement. The Produce Phase encompasses the cleanup that happens prior to initially sharing our results to a broader community beyond our team, for example, ahead of submitting our work to peer review. It also includes the process of incorporating suggestions for improvement prior to finalization, for example, adjustments to address reviewer comments ahead of publication. The research products that emerge from a given workflow may vary in both their form and their formality—indeed, some research products, like a code base, might continually evolve without ever assuming “final” status—but each product constitutes valuable contributions that push our field’s scientific boundaries in their own way.Importantly, producing public-facing products over the course of an entire workflow  rather twhite papers, and preprints. Researchers would also be well advised to register for a unique and persistent digital identifier to be associated with their name, called an ORCID iD (https://orcid.org), as an additional method of tracking and attributing their personal outputs over the course of their career.Building our data science research portfolio requires a method for tracking and attributing the many products that we might develop. One important method for tracking and attribution is the digital object identifier or DOI. It is a unique handle, standardized by the International Organization for Standardization (ISO), that can be assigned to different types of information objects. DOIs are usually connected to metadata, for example, they might include a URL pointing to where the object they are associated with can be found online. Academic researchers are used to thinking of DOIs as persistent identifiers for peer-reviewed publications. However, DOIs can also be generated for data sets, GitHub repositories, computational notebooks, teaching materials, management plans, reports, A third, longer-term goal of the Produce Phase involves establishing a researcher’s professional trajectory. Every individual needs to gauge how their compendium of research products contribute to their career and how intentional portfolio building might, in turn, drive the research that they ultimately conduct. For example, researchers who wish to work in academia might feel obliged to obtain “academic value” from less traditional research products by essentially reprising them as peer-reviewed papers. But judging a researcher’s productivity by the metric of paper authorship can alter how and even whether research is performed . Increashttp://www.datadryad.org) and Figshare (https://figshare.com/) among others.The old adage that one person’s trash is another’s treasure is relevant to the Explore Phase of a data science analysis: Of the many potential applications for a particular data set, there is often only time to explore a small subset. Those applications which fall outside the scope of the current analysis can nonetheless be valuable to our future selves or to others seeking to conduct their own analyses. To that end, the documentation that accompanies data exploration can furnish valuable guidance for later projects. Further, the cleaned and processed data set that emerges from the Explore Phase is itself a valuable outcome that can be assigned a DOI and rendered a formal product of this portion of the data analysis workflow, using outlets like Dryad (https://dwc.tdwg.org) for biodiversity data and EML (https://eml.ecoinformatics.org) for ecological data. To maximize the utility of a publically accessible data set, during the Produce Phase, researchers should confirm that it includes descriptive README files and field descriptions and also ensure that all abbreviations and coded entries are defined. In addition, an appropriate license should be assigned to the data set prior to publication: The license indicates whether, or under what circumstances, the data require attribution.Publicly sharing the data set, along with its metadata, is an essential component of scientific transparency and reproducibility, and it is of fundamental importance to the scientific community. Data associated with a research outcome should follow “FAIR” principles of findability, accessibility, interoperability, and reusability. Importantly, discipline-specific data standards should be followed when preparing data, whether the data are being refined for public-facing or personal use. Data-intensive researchers should familiarize themselves with the standards relevant to their field of study and recognize that meeting these standards increases the likelihood of their work being both reusable and reproducible. In addition to enabling future scientists to use the data set as it was developed, adhering to a standard also facilitates the creation of synthetic data sets for later research projects. Examples of discipline-specific data standards in the natural sciences are Darwin Core (https://gist.github.com/). Tools such as Dr.Watson (https://github.com/JuliaDynamics/DrWatson.jl) and Snakemake [The Git repositories or computational notebooks that archive a data scientist’s approach, record the process of uncovering coding bugs, redundancies, or inconsistencies and note the rationale for focusing on specific aspects of the data are also useful research products in their own right. These items, which emerge from software development practices, can provide a touchstone for alternative explorations of the same data set at a later time. In addition to documenting valuable lessons learned, contributions of this kind can formally augment a data-intensive researcher’s registered body of work: Code used to actively clean data or record an Explore Phase process can be made citable by employing services like Zenodo to add a DOI to the applicable Git commit. Smaller code snippets or data excerpts can be shared—publicly or privately—using the more lightweight GitHub Gists  are marked as such.Potential Products in the Explore PhasePublication of cleaned and processed data set (DOI)Citable GitHub repository and/or computational notebook that shows data cleaning/processing, exploratory data analysis.  (DOI)GitHub Gists White paper Blog post Teaching/training materials Preprint  (DOI)Peer-reviewed publication  (DOI)Potential Products in the Refine PhaseWhite paper Citable GitHub repository and/or computational showing methodology and results (DOI)Blog post Teaching/training materials Preprint  (DOI)Peer-reviewed publication  (DOI)Grant application incorporating the data management procedureMethodology  (DOI)This might include a package, a library, or an interactive web application.See A software toolIn the Refine Phase, documentation and the ability to communicate both methods and results become essential to daily management of the project. Happily, the implementation of these basic practices can also provide benefits beyond the immediate team of research collaborators: They can be standardized as a Data Management Plan or Protocol (DMP). DMPs are a valuable product that can emerge from the Refine Phase as a formal version of lessons learned concerning both research and team management. This product records the strategies and approaches used to, for example, describe, share, store, analyze, and preserve data.While DMPs are often living documents over the course of a research project, evolving dynamically with the needs or restrictions that are encountered along the way, there is great utility to codifying them either for our team’s later use or for others conducting similar projects. DMPs can also potentially be leveraged into new research grants for our team, as these protocols are now a common mandate by many funders . The groRio Journal). Importantly, when new members join a research team, they should receive a copy of the group’s DMP. If any additional training pertinent to plans or protocols is furnished to help get new members up to speed, these materials too can be polished into research products that contribute to scientific advancement. For a list of potential research product examples for the Refine Phase, see As with the research products that are generated by the Explore Phase, DMPs can lead to polished blog posts, training materials, white papers, and preprints that enable researchers to both spread the word about their valuable findings and be credited for their work. In addition, peer-reviewed journals are beginning to allow the publication of DMPs as a formal outcome of the data analysis workflow  that allows the easy exploration of cleaned data sets or demonstrates the outcomes of alternative research questions. More complex examples include a software package that builds an open-source analysis pipeline or a data structure that formally standardizes the problem space of a domain-specific research area. In all cases, the README files, docstrings, example vignettes, and appropriate licensing relevant to the Explore phase are also a necessity for open-source software. Developers should also specify contributing guidelines for future researchers who might seek to improve or extend the capabilities of the original tool. Where applicable, the dynamic equations that inform simulations should be cited with the original scientific literature where they were derived.One very simple example of a tool might be an interactive web application built in RShiny ?Domain expertise?Programming expertise?Collaborative research partners with either time, funding, or relevant expertise?Will the process of creating the new tool be valued/helpful for your career goals?Do we have the resources required to develop a new tool?Should I make a tool?Should we build on an existing tool or make a new one?What research area is it designed for?Who is the envisioned end user? What is the goal of the end user? What is the scope of a new tool?What tool should we create?What are field norms?Is it accessible ?Language choiceWhat is the likely form and type of data input to our tool?What is the desired form and type of data output from our tool?Are there preexisting structures that are useful to emulate, or should we develop our own?Data structures and typesIs there an existing package that provides basic structure or building block functionalities necessary or useful for our tool, such that we do not need to reinvent the wheel?Platform or frameworkHow should we structure the tool?Defining principles for data analysis workflows is important for scientific accuracy, efficiency, and the effective communication of results, regardless of whether researchers are working alone or in a team. Establishing standards, such as for documentation and unit testing, both improves the quality of work produced by practicing data scientists and sets a proactive example for fledgling researchers to do the same. There is no single set of principles for performing data-intensive research. Each computational project carries its own context—from the scientific domain in which it is conducted, to the software and methodological analysis tools we use to pursue our research questions, to the dynamics of our particular research team. Therefore, this paper has outlined general concepts for designing a data analysis such that researchers may incorporate the aspects of the ERP workflow that work best for them. It has also put forward suggestions for specific tools to facilitate that workflow and for a selection of nontraditional research products that could emerge throughout a given data analysis project.Aiming for full reproducibility when communicating research results is a noble pursuit, but it is imperative to understand that there is a balance between generating a complete analysis and furnishing a 100% reproducible product. Researchers have competing motivations: finishing their work in a timely fashion versus having a perfectly documented final product, while balancing how these trade-offs might strengthen their career. Despite various calls for the creation of a standard framework ,46, achiIn addition to amassing outputs beyond the peer-reviewed academic publication, there are increasingly venues for writing less traditional papers that describe or consist solely of a novel data set, a software tool, a particular methodology, or training materials. As the professional landscape for data-intensive research evolves, these novel publications and research products are extremely valuable for distinguishing applicants to academic and nonacademic jobs, grants, and teaching positions. Data scientists and researchers should possess numerous and multifaceted skills to perform scientifically robust and computationally effective data analysis. Therefore, potential research collaborators or hiring entities both inside and outside the academy should take into account a variety of research products, from every phase of the data analysis workflow, when evaluating the career performance of data-intensive researchers .


We aimed to develop a clinical applicable nomogram to predict overall survival (OS) for patients with curatively resected nonmetastatic colorectal cancer.Records from a retrospective cohort of 846 patients with complete information were used to construct the nomogram. The nomogram was validated in a prospective cohort of 379 patients. The performance of the nomogram was evaluated with concordance index (c‐index), time‐dependent receiver operating characteristic (ROC) curves, calibration plots, and decision curve analyses for discrimination, accuracy, calibration ability, and clinical net benefits respectively, and further compared with AJCC 8th TNM staging and the MSKCC nomogram. Risk stratification based on nomogram scores was performed with recursive partitioning analysis.P < .05).The nomogram incorporated age, Glasgow prognostic score, pretreatment carcinoembryonic antigen levels, T staging, N staging, number of harvested lymph nodes, and histological grade. Compared with the 8th AJCC staging and MSKCC model, the nomogram had a statistically higher c‐index , bigger areas under the time‐dependent ROC curves , and improved clinical net benefits. Calibration plots revealed no deviations from reference lines. All results were reproducible in the validation cohort. Nomogram‐based risk stratification successfully discriminated patients within each AJCC stage . The nomogram outperformed the 8th AJCC staging and the MSKCC model and could aid in personalized treatment and follow‐up strategy for CRC patients. We developed and validated a prognostic nomogram for non‐metastatic colorectal cancer incorporating routinely available factors. Nomogram based risk stratification successfully discriminated patients within each AJCC stage. The nomogram outperformed the 8th AJCC staging and the MSKCC model. They utilize computational integration of multiple prognostic factors to quantify risks individually, rather than produce risk groups.Compared to tumor‐related factors, patient factors draw less attention. However, patient factors, for example, age, systemic inflammatory status, and nutritional status, are equally contributed to patients' prognosis.The goal of this project is to develop and assess a prognostic nomogram for curatively resected CRC by incorporating clinical available tumor‐ and patient‐related factors. We hope such a tool could help physicians to convey individualized survival information to every patient in daily practice without incurring additional cost.22.1The data of patients with surgically treated nonmetastatic CRC patients were retrieved from a prospectively maintained cancer registry database of affiliated hospital of Jiangnan University as previously mentioned.Patients who received curative CRC surgery between 2008 and 2013 were as the primary cohort and those treated between 2014 and 2015 as the validation cohort. The inclusion criteria for the primary cohort were as follows: (a) Patients who had curative resection of primary CRC malignancies. (b) Who had histologically confirmed colorectal adenocarcinoma. (c) Who had the full blood cell count, biochemical profile, and tumor biomarker test at the hospital within 2 weeks before surgery. Patients with any of the following conditions were excluded: (a) Who had either imaging or histologically confirmed metastatic CRC diagnosed either preoperatively or intraoperatively. (b) Who had metastatic disease within 1 month after surgery. (c) Who had neo‐adjuvant chemotherapy, radiotherapy, or targeted therapy. (d) Who had bowel obstruction or perforation with emergency presentation. (e) Who was complicated with other acute diseases such as pneumonia, urinary tract infection, and cholecystitis. (f) Who had a history of chronic inflammatory diseases such as inflammatory bowel diseases and rheumatoid arthritis. (g) Who had a previous history of malignancies including CRC at different sites. (h) Who died within 1 month after surgery. (i). Whose survival status could not be ascertained. (j) Whose number of sampled lymph nodes was below 12.The information for potential prognostic variables was collected: demographic characteristics including age and sex; pathological characteristics including primary site of tumor, histology, depth of primary tumor invasion (T), number of total lymph nodes sampled (TLN), number of metastasized lymph nodes (LNM), histological grade (G1‐4), the presence of peri‐neural invasion (PNI) and lymph‐vascular invasion (LVI) and number of tumor deposits (TDs); blood biomarkers including carcinoembryonic antigen (CEA), white blood cell count, neutrophil count, lymphocyte count, albumin, and C‐reactive protein (CRP). GPS was derived as previously stated.Informed consent to the usage of social‐demographic and clinical information in scientific endeavors was obtained from every participant. This study was approved by the ethics review board of the hospital, adhered to the Declaration of Helsinki for medical research involving human subjects, and conducted according to the TRIPOD statement.2.2The endpoint for this study was overall survival (OS), which was defined as the time from the date of surgery to the date of death from any cause or the last date of follow‐up. Patients were censored if they were diagnosed with second malignancies after surgery. Only cases with complete information were used in the final analysis.P < .2) in a backward stepwise selection with minimal AIC (the Akaike information criterion) value. Nomogram based on the final model was constructed for the likelihood of overall survival at 3 and 5 years of surgery.Normally distributed continuous variables were described as mean with standard error (SD), otherwise median values with interquartile ranges (IQR). Cox proportional hazards regression modeling was used to assess the relationship of OS with predictive variables. For continuous variables, possible nonlinearity effects on the log relative hazard of outcome were tested by modeling with restricted cubic splines, whereas statistically significant nonlinearity was identified, restricted cubic splines were used in the multivariable modeling. If restricted cubic spline modeling was failed, continuous variables were dichotomized, for which the optimal cut‐points were determined by the maximally selected rank statistics to maximize the correlation with survival. The proportional hazards assumption for each variable was checked by the test proposed by Grambsch and Therneau. Multivariate models were built by including all variables from univariate models . External validation was conducted in the prospective validation cohort. Briefly, the validation cohort was individually given a risk score calculated with the nomogram equation.The performance of the nomogram was assessed and compared with the MSKCC modelP < .05 as significant and conducted with Stata 14 or R studio software (version 1.1.456).All statistical analyses were two sided with 33.1A total of 1576 patients were initially screened for enrollment eligibility in the study. After application of exclusion criteria, 836 patients with complete information were included in the final analysis for the primary cohort. For the validation cohort, a total of 379 patients of 505 patients were included in the final analysis after application of the same inclusion and exclusion criteria. The major reason for exclusion was less than 12 lymph nodes sampled, followed by metastasized disease and preoperative treatment. The percentage of patients excluded for missing data was less than 10%.The baseline characteristics of the primary cohort and validation cohort were listed in Table 3.2P = .016 and <.001, respectively). Restricted cubic spline modeling was applied to all other continuous variables with nonlinear effects, except for CEA. CEA more than 20.03 ng/mL was regarded as high CEA level, otherwise as low. The results of univariate cox regression survival analysis for the primary cohort were presented in Table Among continuous variables, only age and LNM had linear effects . For the external validation, the score for the individual case in the validation cohort was calculated according to the established nomogram and was then used in the Cox regression model. C‐index for the validation cohort was 0.79 (95% CI: 0.73‐0.85).3.4The calibration plots showed a good agreement between observed and nomogram predicted 3‐ Figure  and 5‐ye3.5The nomogram had a statistically higher c‐index compared with the MSKCC model and AJCC staging in both the primary cohort and validation cohort. The c‐indexes were 0.77 (95% CI: 0.73‐0.80), 0.73 (95% CI: 0.70‐0.76), 0.69 (95% CI: 0.66‐0.92) for the nomogram, MSKCC model, and AJCC staging respectively in the primary cohort and 0.79 (95% CI: 0.73‐0.85), 0.75 (95% CI: 0.68‐0.81), 0.68 (95% CI: 0.61‐0.75) in the validation cohort. The nomogram had the lowest AIC value among the three models .P < .05, all pair‐wise comparisons)    Figure . In the ) Figure .Decision curve plots showed the nomogram was associated with improved clinical net benefits over the MSKCC model and AJCC stages (higher lines of prediction by the nomogram) within a practical range of threshold probabilities in both the primary cohort Figure  and the 3.6Patients were stratified into three risk groups based on nomogram derived risk scores with a recursive partitioning analysis. The subgroups were as follows: low‐risk group (risk score ≤12.68), intermediate‐risk group (12.68 < risk score ≤ 14.07), and high‐risk group (risk score >14.07). Kaplan‐Meier survival curve analysis showed the three groups had statistically different prognosis in both primary cohort Figure  and vali4In the present study, a nomogram was developed to estimate 3‐year and 5‐year survival probability for curatively resected nonmetastatic CRC patients. This nomogram outperformed the MSKCC model and AJCC TNM staging in terms of discrimination, calibration abilities, and clinical utilities. The nomogram was validated in a prospective cohort and demonstrated to be quite reliable.The selection of factors in this study was based on their availability in routine practice and established associations with overall survival in previous publications.To date, there are four published nomograms predicting survival after radical surgery for nonmetastatic CRC. Two were developed from SEER database.The strengths of our nomogram include an appreciable size of representative patients in real clinical setting, a prospective validation cohort and readily available factors in routine practice. The variables used in the nomogram could be easily obtained by physicians in many community hospitals without any technical or cost barriers. Risk group stratification defined by the nomogram was a good complement to the 8th AJCC stage. The nomogram gives accurate and individualized mortality risk predictions and can discriminate different prognosis groups within the same TNM stage. It should enable improved patient counseling regarding treatment selection and follow‐up strategy. The nomogram itself is not intended to make treatment decisions, but the nomogram directed treatment strategy could be investigated in clinical trials.There are several points should be addressed. First and foremost, this nomogram was developed from a cohort of patients treated at a single institution including only Chinese patients. Although internal validation and prospective external validation were performed to prevent over‐fit of current data, it would be better to validate the nomogram in patients from other institutions with diversified ethnicities. Second, important molecular factors such as KRAS/NRAS/BRAF and MSI were not investigated. These factors were good treatment efficacy predictors, but their prognostic roles were controversial. In a recent meta‐analysis, they were found to be not significantly or differentially associated with survival.5In conclusion, we propose a nomogram that could provide individualized outcome predictions with good accuracy, reliability, availability, and applicability. It could be helpful to physicians and patients in the treatment decision‐making process.All authors declare no conflict of interest.Tingting hong and Xiaohong Wu designed the study. Tingting hong analyzed and interpretated the data and wrote the manuscript. Dongyan cai and Ying zhang collected the data. Dong hua, Linfang Jin, and Tingxun Lu provided valuable insights into data interpretation and manuscript writing.


Subclinical, low-grade, inflammation is one of the main pathophysiological mechanisms underlying the majority of chronic and non-communicable diseases. Several methodological approaches have been applied for the assessment of the anti-inflammatory properties of nutrition, however, their impact in human body remains uncertain, because of the fact that the majority of the studies reporting anti-inflammatory effect of dietary patterns, have been performed under laboratory settings and/or in animal models. Thus, the extrapolation of these results to humans is risky. It is therefore obvious that the development of an inflammatory model in humans, by which we could induce inflammatory responses to humans in a regulated, specific, and non-harmful way, could greatly facilitate the estimation of the anti-inflammatory properties of diet in a more physiological way and mechanistically relevant way. We believe that exercise-induced muscle damage (EIMD) could serve as such a model, either in studies investigating the homeostatic responses of individuals under inflammatory stimuli or for the estimation of the anti-inflammatory or pro-inflammatory potential of dietary patterns, foods, supplements, nutrients, or phytochemicals. Thus, in this review we discuss the possibility of exercise-induced muscle damage being an inflammation model suitable for the assessment of the anti-inflammatory properties of diet in humans. Subclinical, low-grade, inflammation is one of the main pathophysiological mechanisms underlying the majority of chronic and non-communicable diseases ,2,3,4,5.The inflammatory responses are integral parts of the normal innate immune response conferring protection to infection and initiating mechanisms of repair and regeneration of damaged tissues ,21. UndeIt is well documented that a subclinical activation of the inflammatory mechanisms may be a predisposing risk factor for non-communicable diseases such as cardiovascular disease, cancer, metabolic syndrome, diabetes, depression, dementia, and biological aging in general ,34,35,36It is now known that prudent dietary patterns, such as the Mediterranean Diet ,39,44,45Several methodological approaches have been applied for the assessment of the anti-inflammatory properties of nutrition. The majority of them is based on cellular and animal models of inflammation. Cellular models of inflammation include LPS-induced secretion of cytokines, expression of adhesion molecules in the surface of cells, phagocytosis activity, natural killer cells lysing capability against cancer cells etc. Cell-based assays were mainly utilized for the identification of the anti-inflammatory properties of isolated nutrients, extracts, and phytochemicals. Despite their usefulness for screening purposes and mechanistic studies the results of those studies cannot be extrapolated directly to humans ,13,40,46The rapid growth of genetic engineering enabled the development of a plethora of animal models of inflammation by which the anti-inflammatory properties of diet could be assessed, in a more physiological way. Moreover, animal experiments provide wider access to the immune system . However, after many years of studying and working with animals in biomedical research, ethical issues have emerged concerning the reproducibility of animal models and their relevance with human inflammatory diseases ,40,46,48The majority of human studies, investigating the association between diet and inflammation, are cross-sectional and prospective epidemiological studies ,52,53,54EIMD is a phenomenon that occurs either after a prolonged unaccustomed exercise, or after a very intense, high-demanding exercise , as a consequence of the very high mechanical and metabolic demands during the exercise ,73,74,75The type of exercise and the type of muscle contractions applied are crucial determinants of the damaging and inflammatory responses. It is well established that the eccentric exercise, which involves only lengthening muscle actions, could lead to extensive EIMD and inflammation ,86,87,882+ concentrations in the cytoplasm lead to degradation of muscle proteins and membrane phospholipids by activation of calpains and phospholipases A2 [The inflammatory response, after EIMD, has been characterized quite well and several excellent original and review articles describe it in detail ,93,119. pases A2 . MeanwhiTheir main purpose is to degrade the damaged tissue by releasing active oxygen/nitrogen forms, and induce oxidative stress which is crucial for muscle regeneration after EIMD ,123,124.EIMD can be easily accepted by the volunteers and the bioethics committees. It can be easily induced by different types of exercise/training either in the lower or the upper limbs (see below). The inflammatory response can be applied to all kinds of populations irrespective of their health and training status, age, gender, race, body composition etc. Most importantly, volunteers easily consent to this kind of intervention which is actually just about exercise for them. In addition, most bioethics committees would have no objections to this kind of experimentation in humans.EIMD can be easily applied to humans in a regulated manner. Exercise scientists can induce muscle damage by forcing muscles to lengthen while generating active tension, with various stimuli. As it has been discussed in the previous section, EIMD can be easily induced through eccentric training after 85 to 300 maximal eccentric contractions, while the magnitude and the extent of EIMD from eccentric exercise, seems to be higher and prolonged (lasting for 72 h until 1–2weeks) compared to other type of exercises ,199,200.The inflammatory mechanisms underlying EIMD are well defined. The muscle microtrauma, induced by the different types of exercise, but mostly from eccentric exercise, can trigger a typical cascade of inflammatory events that resemble aseptic inflammation after tissue damage (see above). It is therefore easier for researchers to identify the crucial mechanistic points that each intervention could affect.One of the biggest advantages of EIMD, is that researchers could have the whole picture of the inflammatory response and its lysis in a strict and regulated time course. In contrast, when individuals with already established low-grade, chronic inflammation are recruited, the variability of the clinical and biochemical phenotypes, pharmacology, and medical history is usually large even in well-controlled studies. Thus, even in the best controlled cross sectional studies, the diversity between the participants would have a strong conflicting impact on research outcomes.Biological sampling. Apart from the classical blood or saliva samples, that are usually collected before and several time points after the exercise trial this type of experiments allow you to take samples of the inflamed tissue, namely muscle biopsies. This technique has been used in many studies, investigating either the training-induced adaptations on muscle fibers The kinetics of clinical phenotypes linked to the inflammatory response can be easily determined. Such phenotypes are delayed-onset muscle soreness, maximum isometric torque, range of motion, limb circumference, and several other types of ergometric tests according to the inflamed limb.In our opinion, EIMD is a convenient, dynamic, and informative model of inflammation. It can be applied either in studies investigating the homeostatic responses of individuals under inflammatory stimuli or for the estimation of the anti-inflammatory or pro-inflammatory potential of dietary patterns, foods, supplements, nutrients or phytochemicals. The main advantages of EIMD are the following :EIMD canAcute and chronic inflammations. EIMD inflammatory response share similar pathophysiological and biochemical responses with acute and chronic inflammation. Thus, EIMD could find application in studies investigating the effect of nutrition and/or exercise, in acute and chronic inflammatory conditions such as those observed before and/or during the majority of chronic and non-communicable diseases ,128,272.Considering the pathophysiology behind conditions such as muscle/neurogenic inflammation, atrophy, cachexia, sarcopenia, and chronic muscle protein degradation EIMD can serve as a very reliable and regulated model to investigate how nutrition and or exercise may affect the physiological and biochemical background of those conditions.Ischemic preconditioning (IPC). After an EIMD stimuli, the following exercise bouts induce lower muscle damage and inflammation, due to the specific muscle adaptations, that minimize the extent of muscle damage, a phenomenon that it is known as “repeated bout effect” RBE ,275,276),276275,2Rhabdomyolysis. Rhabdomyolysis is a pathophysiological condition of extensive skeletal muscle cell damage which could be induced from many physical  and non-physical causes  . This isFibromyalgia (FM). FM is a complex syndrome characterized by widespread pain that affects many tissues and the presence of allodynia and hyperalgesia ,287. FatThe EIMD model can serve as a precise model, in studies investigating the effects of nutritional and training interventions, on acute and chronic inflammation conditions, mainly in metabolic-related inflammatory conditions. As for example, EIMD could serve as a useful model in human studies investigating:According to the above, it seems that EIMD is a controlled, highly regulated tool in the hands of researchers, to investigate the ability of different types of nutritional and training interventions to interfere with the progression and lysis of acute inflammatory conditions. It is worth testing if the ability of those interventions to attenuate acute inflammatory responses after EIMD can predict their ability to also act protectively against chronic inflammation although such as link has not been established yet. Nevertheless, EIMD allows the assessment of the putative anti-inflammatory, hermetic, or immunomodulatory properties of dietary intervention directly to human beings under real, dynamic conditions.


Mesenchymal stem cells (MSCs) have been identified as potential therapeutic candidates for cardiovascular disease (CVD). However, the molecular mechanism underlying their therapeutic effects is unknown.Recently, Chen et al. showed that an exosomal lncRNA derived from MSCs protected against cardiomyocyte apoptosis in vivo, which provided new insight into our understanding of the functions of MSCs. Other exosomal lncRNAs from MSCs have been shown to play pivotal roles in the development of CVD.Exosomal lncRNAs are important molecules in the application of MSCs, which reveals, at least in part, the functional mechanism of MSCs. Dear Editor,Cardiovascular disease (CVD) is the leading cause of morbidity and mortality worldwide. Mesenchymal stem cells (MSCs) were identified as potential therapeutic candidates for CVD many years ago. Delivery of MSCs decreased infarcted size, improved cardiac function, regulated cardiac cell function, and provided a histological network for cell attachment. However, the underlying molecular mechanism remains to be elucidated.2O2-induced apoptosis in cardiomyocytes by modulating the expression of miR-142-3p and activating Forkhead class O1 (FOXO1). This finding suggests the potential of exosomal lncRNA-mediated therapy for clinical applications [Exosomes have recently been shown to be important mediators of cell-to-cell communication. Exosomes are lipid bilayer-enclosed nanovesicles containing multiple molecules, including proteins, DNAs, and RNAs, that can transfer biological information under both normal physiological and pathological conditions. Recently, Chen et al. reported that exosomal long non-coding RNA (lncRNA) derived from MSCs protected cardiomyocytes against apoptosis in vivo, which provided new insight into our understanding of the functional role of MSCs. According to the study, lncRNA-NEAT1 derived from exosomes prepared from macrophage migration inhibitory factor (MIF)-induced MSCs blocked HSimilar to lncRNA-NEAT1, other MSC-derived exosomal lncRNAs showed activity in CVD. Exosomal metastasis-associated lung adenocarcinoma transcript 1 (MALAT1), derived from human umbilical cord MSCs, blocked aging-induced cardiac dysfunction by inhibiting the nuclear factor kappa B (NF-κB)/tumor necrosis factor-α (TNF-α) signaling pathway . In addiTaken together, these findings demonstrate that exosomal lncRNAs are important molecules in the application of MSCs, and at least partially reveal the functional mechanism underlying the effects of MSCs. Nevertheless, our understanding of the precise regulatory mechanisms of exosomal lncRNAs in different cells is still in its infancy. For example, while it is known that some lncRNAs are expressed at low levels in cells but are enriched in secreted exosomes, how the exosome “captures” these lncRNA molecules is yet unknown. Although lncRNAs may have a better chance of escaping ribonuclease degradation through their association with exosomes, it is not known whether lncRNAs are modified within the exosome. In addition, lncRNAs, miRNAs, and mRNAs can be regulated by other molecules, which makes determining the regulatory relationships challenging. Therefore, further study is needed to fully understand the characteristics of exosomal lncRNAs.