Task: Sentence Composition
Input (English): Access to an interactive site for purposes of computing a score predicting one’s risk of dying will be free. However, this should have been stated in the release. We can’t rate the release Satisfactory if there’s no mention of cost at all, so we’ll rate it Not Applicable. Very little about this news release is quantitative. The document repeatedly lauds the benefits of predicting 5-year mortality via self-reports rather than more intrusive physical tests, and it touts the potential use of the “Ubble age” score to assist doctors in identifying “high risk” patients and to improve individuals’ self-awareness. But no studies have been conducted to test effects on either of these audiences, a concern which the release refers to only at the end of the document. The release could have included that information higher up in the release and emphasized it more. The release does mention that the algorithm was validated and shown to have an ~80% accuracy. But as noted above, this is a population level estimate. The overall test characteristics of the model do not translate into an interpretation that is meaningful for an individual. One could argue that this is mainly the fault of the original study rather than the release, as we could find no data in the paper that provides information that an individual could use to determine how accurate their particular calculation was. However, this problem contributes to an overall sense that the release is weighted too far in the direction of benefits without enough attention to potential harms and uncertainties. The text does offer a brief cautionary note when it quotes from the commentary of a couple of UK scientists in The Lancet issue to the effect that “whether this will help individuals improve self-awareness of their health status…or only lead to so-called cyberchondria, is a moot point.” Cyberchondria refers to unfounded anxiety concerning the state of one’s health brought on by visiting health and medical websites, so that term seems apt here. And “moot” conveys the uncertainty surrounding outcomes as, again, the study can offer no evidence. We’ll give a Satisfactory rating, though we wonder if it would have been more effective to say, “it is unclear whether use of this information will lead to actions that will help or harm individuals.” The data used came from a gargantuan epidemiological study involving nearly half a million UK respondents. The news release offers some details about that study, including the number of possible predictors (655) and the nature of the analysis. The text also contains a caveat reminding the reader that prediction is not the same as causation (a variant on the “correlation is not causation” theme), as well as a quote from one of the investigators that cautions against seeing one’s score as a “deterministic prediction.” These are important qualifiers, but they occur lower in the press release, and it is not clear how well those points will get picked up in subsequent stories. We didn’t see anything that implied a doomsday perspective to use of this data. The release also clearly implies this is for UK citizens and not for others. So we’ll award a Satisfactory rating, although we’d note again that there’s an aspect of this test that could possibly lead to increased anxiety and “cyberchondria,” as the release puts it. Funders are clearly identified in the notes at the bottom of the press release. The investigators assert the absence of conflicts of interest in the journal article itself. The study and its press release make much of the potential advantage of predicting death via self-report questions rather than biological tests. So the text makes an overt comparison, admittedly without data. The release could also have included the comparison to doing nothing. Whether providing any risk information (this version or another) or no risk information makes a difference is a legitimate question. Notes at the end of the news release speak to the availability of the interactive “Ubble” website once the research article is published. We could not access the calculator, perhaps due to high traffic levels accompanying the rollout. The news release is explicit about the novelty of the approach to prediction, as well as regarding the scale of the study itself. We suspect it is correct in this assessment, as the epidemiological study on which the work is based is truly huge. One of the investigators refers to the study’s development of a prediction score from self-report data as “exciting,” but most of the quotes, while positive, also convey useful information, including qualifiers.
Output (English): Simple score predicts risk of death for middle-aged adults in the UK