trentmkelly/LessWrong-43k huggingface dataset:
An X-Ray is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation
by hugofry — 2024-10-07T08:53:14.658Z
Link: https://arxiv.org/abs/2410.03334

This is an archival link-post for our preprint, which can be found here.

Figure 1: SAE-Rad identifies clinically relevant and interpretable features within radiological images. We illustrate a number of pathological and instrumentation features relevant for producing radiology reports. We add annotations (green arrows) to emphasize the presence of each feature.

Executive Summary

This post is a heavily slimmed down summary of our main paper, linked above. We have omitted all the technical details here. This post acts as a TL;DR archival link-post to the main paper.

We train Sparse Autoencoders (SAEs) on the class token of a radiology image encoder, on a dataset of chest x-rays. We use the trained SAE, in conjunction with automated interpretability, to generate radiology reports. The final radiology report represents a concatenation of the text descriptions of activate SAE features. We train a diffusion model to allow causal interventions on SAE features. This diffusion model enables us to highlight where in the chest x-ray each sentence in the radiology report comes from by localising changes in the image post-intervention. Our method achieves competitive accuracy in comparison to state of the art medical foundation models while using a fraction of the parameter count and compute costs. To the best of our knowledge, this is the first time SAEs have been used for a non-trivial downstream task - namely to perform multi-modal reasoning on medical images.

Of particular note to the mechanistic interpretability community, we demonstrate that SAEs extract sparse and interpretable features on a small dataset (240,000) of homogenous images (chest x-rays appear very homogenous), and that these features can be accurately labeled by means of automated interpretability to produce pathologically relevant findings.

Motivation

Radiological services are essential to modern clinical practice, with demand rising rapidly. In the UK, the NHS performs over 43 million radiological procedures annually, costing over £2 billion, and demand for scans more than doubled between 2012 and 2019. A significant portion of these costs addresses rising demand through agency, bank, and overtime staff, but a national imaging strategy notes this funding is unsustainable. Consequently, there’s growing interest in (semi)-automating tasks like radiology report generation, augmentation, and summarization to ...
Compelling Villains and Coherent Values
by Cole Wyeth — 2024-10-06T19:53:47.891Z

Epistemic status: This essay attempts to communicate intuitions strong shaped by @Joe Carlsmith's writing, particularly "On Green."

Warning: very minor spoilers, mostly for "The Dark Knight".

Some people have a sense of weight about them. They know exactly what they want and can explain why it is Right. Presented with a moral dilemma, they resolve conflicts between local values in a consistent way by appealing to higher values - they can tell you today whether they would actually pull the lever in the trolley problem and why they (hopefully) wouldn't harvest the organs from one patient to save five - or why they would be compelled to. Most of these people seem to be deeply religious - the real-world example I have in mind is an intelligent Catholic friend who once had his bike stolen and explained to me that it was a shame because he would have happily given it away if someone needed it. I know him well enough to say it was true. Ask him what he wants out of life and he gives a well-reasoned, specific answer justified in terms of a hierarchy of values from family and charity to (his idea of) "serving God." I think of this as moral[1] coherence.

The paragon of this class is some sort of warrior monk defending the gates of an abbey against a horde of barbarians, and afterwards burying each of them with an hours long prayer. The opposite is perhaps the Joker, who we will come back to.

Mostly, I don't get this sense of weight from nerds or rationalists - I think it's not a natural part of nerd culture. Nerds are too clever and contrarian; we mostly can't even agree with ourselves. Also, we tend to "throw off the shackles" of religion and often conventional morality, which are both systems carefully engineered for moral coherence, though modern morality is much more cosmopolitan. However, I do get a certain sense of moral coherence off of Eliezer (Scott seems more complicated).

Some of the most compelling villains are morally coherent. They have some consistent, authentic beliefs that set them on a collision course with the protagonist and in the best cases bring out the moral conflicts within the protagonist, often by posing some kind of ultimatum (for instance Jason Bourne, who has a complicated relationship with authority and nationalism, usually fights hardcore patriots). We tend to like a little moral confusion in our protagonists; perhaps it makes the...
To Be Born in a Bag
by Niko_McCarty — 2024-10-06T17:21:00.605Z
Link: https://www.asimov.press/p/artificial-wombs

Written by Tom Ireland & published by Asimov Press.

The womb is a remarkable organ — a muscular, pear-shaped chamber that supports the transformation of a tiny cluster of dividing cells into an entirely new person. All humans begin their lives in this sturdy chamber.

At least for now.

Several research groups are busily developing artificial wombs to replicate the basic life-support functions of the uterine environment for extremely premature infants — that is, babies born at or before 28 weeks of the typical 37-42 week pregnancy.[1]

At least one group is working on something altogether more radical: a system to grow a fetus from embryo to birth entirely outside the womb, a concept known as ectogenesis. And while currently being developed under the guise of helping to rescue extinct or critically endangered species, such technology could have profound implications for the future of human reproduction.

In 2018, researchers at the Children’s Hospital of Philadelphia managed to

support the growth of fetal lambs for up to 4 weeks

outside the womb,[2]using an artificial womb to replicate the gaseous exchange function of the placenta, oxygenating the fetus’ blood and removing carbon dioxide. Results such as these have been so encouraging that in September of last year, the FDA began to

review the idea of moving to human trials

.

Despite this enthusiasm, however, a number of technological challenges remain before scientists can test such wombs on human babies. Premature infants differ from animal fetuses in both physiology and how willing we are to subject them to risk. Indeed, the stakes in the first real-world use of artificial wombs may be higher than in almost any clinical trial imaginable.

As for the more ambitious idea of growing an animal from embryo to birth outside the womb, challenges remain even to get to animal trials. Scientists must synthetically replicate the process that encourages an embryo to grow a physical connection to its mother, a developmental phase that is still poorly understood. Next, they need to devise an artificial interface that replicates the complexities of the endometrium, the lining of the uterus that connects to the placenta. From there, the system must be able to change scale as the embryo grows from the size of a grain of rice to a wriggling full-term infant. And finally, such an environment must be able to mimic the complex mi...
Whimsical Thoughts on an AI Notepad: Exploring Non-Invasive Neural Integration via Viral and Stem Cell Pathways
by Pug stanky — 2024-10-06T16:37:12.912Z

Abstract

This paper explores the speculative potential of non-invasive neural augmentation through the use of viral vectors, stem cells, and the body’s natural neuroplasticity. Specifically, it hypothesizes the utilization of neurotropic viruses, such as modified herpes simplex virus (HSV), in stimulating the body's repair mechanisms to facilitate neural-lace integration. The proposed approach considers peripheral neuron clusters and spinal pathways as viable gateways for neural enhancement, using a combination of viral infection to activate self-repair, stem cell therapy to fortify neural connections, and immune system modulation to guide the body’s responses. These speculative thoughts aim to probe the boundaries between biotechnology and neuroprosthetics, raising questions about non-invasive approaches to future neural augmentation technologies.

Introduction

The integration of human biology with external neural interfaces remains an evolving and highly anticipated frontier in medical science. Most current efforts to develop brain-computer interfaces (BCIs) and neural lace technologies focus on invasive methods that require direct brain implantation, often risking adverse side effects such as infection, immune rejection, or cognitive disruption.

This paper speculates on an alternative: using viral vectors, stem cells, and the body's own repair mechanisms to facilitate the integration of external interfaces without requiring invasive surgery. Central to this exploration is the idea that neurotropic viruses, like HSV, could be adapted to stimulate local plasticity and repair in neural tissues, allowing the body to "build" its own connections between the brain, peripheral nervous system (PNS), and external interfaces. These ideas extend from current breakthroughs in gene therapy, neuroplasticity, and the regenerative potential of stem cells.

The speculative nature of this discussion, which draws on cutting-edge biotechnological concepts, is meant to provoke dialogue about non-invasive neural augmentation and its ethical, technical, and medical implications.

Viral Vectors: Beyond Infection to Neural Enhancement

Neurotropic viruses, such as HSV, possess a unique ability to infect neurons and remain latent within the nervous system. While commonly associated with negative health outcomes, such as cold sores or encephalitis, these viruses’ ability to bypass the blood-br...
Why I’m not a Bayesian
by Richard_Ngo — 2024-10-06T15:22:45.644Z
Link: https://www.mindthefuture.info/p/why-im-not-a-bayesian

This post focuses on philosophical objections to Bayesianism as an epistemology. I first explain Bayesianism and some standard objections to it, then lay out my two main objections (inspired by ideas in philosophy of science). A follow-up post will speculate about how to formalize an alternative.

Degrees of belief

The core idea of

Bayesian epistemology

: we should ideally reason by assigning credences to propositions which represent our degrees of belief that those propositions are true. (Note that this is different from Bayesianism as a set of statistical techniques, or Bayesianism as an approach to machine learning, which I don’t discuss here.)

If that seems like a sufficient characterization to you, you can go ahead and skip to the next section, where I explain my objections to it. But for those who want a more precise description of Bayesianism, and some existing objections to it, I’ll more specifically characterize it in terms of five subclaims. Bayesianism says that we should ideally reason in terms of:

Propositions which are either true or false (classical logic)
Each of which is assigned a credence (probabilism)
Representing subjective degrees of belief in their truth (subjectivism)
Which at each point in time obey the

axioms of probability

(static rationality)
And are updated over time by applying

Bayes’ rule

to new evidence (rigid empiricism)

I won’t go into the case for Bayesianism here except to say that it does elegantly formalize many common-sense intuitions. Bayes’ rule follows directly from a straightforward Venn diagram. The axioms of probability are powerful and mathematically satisfying. Subjective credences seem like the obvious way to represent our uncertainty about the world. Nevertheless, there are a wide range of alternatives to Bayesianism, each branching off from the claims listed above at different points:

Traditional epistemology

only accepts #1, and rejects #2. Traditional epistemologists often defend a binary conception of knowledge—e.g. one defined in terms of justified true belief (or a similar criterion, like reliable belief).

Frequentism

accepts #1 and #2, but rejects #3: it doesn’t think that credences should be subjective. Instead, frequentism holds that credences should correspond to the relative frequency of an event in the long term, which is an objective fact about the world. For example, you should assign 50%

...
European Progress Conference
by Martin Sustrik — 2024-10-06T11:10:03.819Z

This is a crosspost from https://250bpm.substack.com/p/european-progress-conference

Europe has become known as a hub of degrowth.

Where are the authors of degrowth papers from?

European parliament even hosted a degrowth conference. Ursula von der Leyen addressed the participants and took a conciliatory (although still a pro-growth) stance.

This stuff is scary: I've seen degrowthers going so far as proposing to take lessons from Cuban "urban agriculture", referring to the time when people farmed any available plot of land to get any food at all (this may be familiar to Eastern Europeans who grew potatoes at their dachas in 1990's) and allegedly ate animals in the zoo.

To counter the stigma, it's past the time for Europe to hold a pro-growth counterconference. A conference similar to upcoming Roots of Progress conference, but, given the European specifics, less focused on the theory and and more on just bringing the progress-friendly forces of the continent, across the geopolitical and language divides, into the same room.

Although the situation in Europe may seem bad, especially when comparing it to the US, it's not all doom and gloom.

Europe has great infrastructure, good education, promising research, high human capital (sadly being exported en masse to the US at the moment), it contains the world's largest free market zone, and generally, it has almost all the prerequisites for sustained growth.

To give an example from my own field, software, Europe has slept through the software revolution and realized almost no gains from it.

Yet, at the same time, when you look at the open source software, a substantial part of it comes from Europe. The web was developed by a Briton in Switzerland. Linux was concieved in Finland. Python programming language hails from Netherlands, MySQL from Sweden. My own little contribution to the ecosystem, ZeroMQ, was mostly developed in a basement room in Slovakia. (Yay for central-eastern Europe, the region of crazy politics but a nice growth record!)

Europe hosts the biggest open source conferences, such as FOSDEM (Belgium, 10000 attendees) and CCC (Germany, 17000 attendees).

Moving away from software, Europe has strong pharma (Britain, Switzerland and lately Denmark; check the blogs of Ruxandra Teslo or Alex Telford), Germany and the adjacent regions (northern Italy, Switzerland, Belgium) are still the biggest producers of machine tooling ...
Open Thread Fall 2024
by habryka — 2024-10-05T22:28:50.398Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you're new to the community, you can start reading the Highlights from the Sequences, a collection of posts about the core ideas of LessWrong.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Seeking AI Alignment Tutor/Advisor: $100–150/hr
by MrThink — 2024-10-05T21:28:16.491Z

I am actively looking for a tutor/advisor with expertise in AI x-risk, with the primary goal of collaboratively determining the most effective ways I can contribute to reducing AI existential risks (X-risk).

Tutoring Goals

I suspect that I misunderstand key components of the mental models that lead some highly rational and intelligent individuals to assign a greater than 50% probability of AI-related existential catastrophe ("p-doom"). By gaining a clearer understanding of these models, I aim to refine my thinking and make better-informed decisions about how to meaningfully reduce AI X-risk.

Specifically, I want to delve deeper into why and how misaligned AGI might be developed, and why it wouldn’t be straightforward to solve alignment before it becomes a critical issue.

To clarify, I do NOT believe we could contain or control a misaligned AGI with current safety practices. What I do find likely is that we will be able to avoid a situation altogether.

In addition to improving my understanding of AI X-risks, I also seek to explore strategies that I could aid in implementing in order to reduce AI X-risk.

About Me

- My primary motivation is effective altruism, and I believe that mitigating AI X-risk is the most important cause to work on.
- I have 7 years of experience working with machine learning, with a focus on large language models (LLMs), and possess strong technical knowledge of the field.
- My current p-doom estimate is 25%, derived from my own model, which gives about 5%, but I adjust upward in since some highly rational thinkers predicts significantly higher p-doom. Even if my p-doom were 1%, I would still view AI X-risk as the most pressing issue and dedicate my time to it.

Why Become My Tutor?

- You will be directly contributing to AI safety/alignment efforts, working with someone highly committed to making an impact.
- Opportunity for **highly technical 1-on-1 discussions** about the cutting-edge in AI alignment and X-risk reduction strategies.
- Compensation: $100–150 per hour (negotiable depending on your experience).

Ideal Qualifications

- Deep familiarity with AI existential risks and contemporary discussions surrounding AGI misalignment.
- A genuine interest in refining mental models related to AI X-risk and collaborating on solutions.
- p-doom estimate above 25%, since I aim to understand high p-doom perspectives.
- Strong interpersonal compatibility: It...
Interpretability of SAE Features Representing Check in ChessGPT
by Jonathan Kutasov — 2024-10-05T20:43:36.679Z

Produced by Jon Kutasov and David Steinberg as a capstone project for ARENA. Epistemic status: 5 days of hacking, and there could be bugs we haven’t caught. Thank you to the TAs that helped us out, and to Adam Karvonen (author of the paper our work was based on) for answering our questions and helping us debug early in the project!

Overview

This paper

documents the training and evaluation of a number of SAEs on Othello and

ChessGPT

. In particular, they train SAEs on layer 6 of 8 in these language models. One of the interesting results that they find is that the latents captured by the SAE perfectly/almost perfectly reconstruct several of the concepts about the state of the game.

ChessGPT is an 8 layer transformer model. It reads in PGN strings (that are of the form ‘;1.e4 e5 2.Nf3 …’) and predicts the next token in the sequence. Tokens in this case are single characters, and only those that can possibly appear in a PGN string. Karvonen et al. trained 500 SAEs on the residual stream after layer 6 of this model. Notably, they only used their model to predict the token after a “.” rather than at any point in a PGN string.

We wanted to evaluate if these SAE features are 1. interpretable and 2. relevant to the prediction outputted by the model. Given that chessGPT is reasonably small, chess is a well structured game, and the game comes with nice ground truth labels for different features of the board state, this can be done in a much more efficient way than a similar study on a general LLM.

Specifically, we focused on 30 SAE features that the original authors had found were perfect classifiers (using the metrics they had defined) of the question “is the player to move currently in check?” Our key findings are as follows:

About half of SAE features are assigned to a specific part of the PGN pattern. That is to say, some features are almost entirely focused on predicting the token after a “.” while others are focused on predicting the token after an “x”
As a result, we believe that many of the 30 features identified as good classifiers for check are not actually affecting the model’s prediction in the subset of cases the original paper focuses on (“.”).
Many of the features that are capturing the information “the last token was a ‘.’ and white (the player to move) is in check” can be further described in interpretable ways. For example, a feature may

...
2024 Election Forecasting Contest
by mike20731 — 2024-10-05T20:43:16.203Z
Link: https://www.mikesblog.net/p/announcing-2024-election-forecasting

This is a forecasting contest with cash prizes that I'm running based on 50 questions about the the 2024 election. Posting it here because I think a lot of people in this community like forecasting, prediction markets, etc. (Actually this is my first LW post after years of lurking. Am I allowed to post stuff like this? If I'm breaking a rule on self-promotion then I apologize.)

The primary goal of the contest is just to have fun and let people try their hand at probabilistic forecasting. But a secondary, behind-the-scenes goal is that I'm try to collect crowdsourced forecast data that can be aggregated to generate a "wisdom of crowds" style average forecast, which I'll then compare to the Manifold and Polymarket predictions on the same questions. Basically the idea is to compare 3 incentives types: non-market but cash prize (my contest), play-money market (Manifold) and real-money market (Polymarket).

Deadline is Monday November 4, before 12 noon US Eastern Time. There's also a separate prize for early predictions made by October 22 (2 weeks before election day). Details on scoring methodology in the post. Please consider entering 🙂