trentmkelly/LessWrong-43k huggingface dataset:
Quick look: cognitive damage from well-administered anesthesia
by Elizabeth — 2022-12-02T00:40:01.344Z

Recently a client commissioned me to look at the potential cognitive impacts of general anesthesia. I was surprised to find out that it’s not obvious general anesthesia does more damage than spinal or local anesthesia, and my guess is most but not all of the damage is done by the illness or surgery themselves.

Caveats and difficulties

I’m not a doctor. The following represents something like 5 hours of work, which obviously is not enough time to process even a fraction of the literature. I was focused on the dangers of median uses of anesthesia, where nothing goes obviously wrong and the anesthesiologist considers it a success; I didn’t even attempt to look at the rate of accidents, which can be pretty severe. My friend’s dad’s life was ruined by a fungal contaminant in a spinal injection. And of course, people die from excess general anesthesia. But for this post I only looked at damage done by routine anesthetic usage.

Like all client research, this was tailored to a particular person’s needs and budget, and shouldn’t be considered a general-purpose survey.

It’s pretty hard to tease out the difference between damage done by anesthesia, damage done by whatever necessitated the surgery, and damage done by having your body ripped open and bits moved around. Bodies hate that sort of thing. The few RCTs that exist by necessity focus on a narrow range of minimally invasive surgeries for which there exists a choice in type of anesthesia, and animal studies tended to focus on developing animals rather than adults. Even for procedures where multiple types are possible, patients tend to be pretty opinionated about what they want; one paper even announced they’d given up on reaching their sample size goal because recruiting was too hard.

Studies also often focused on cognition within a few hours of surgery (when people are still at the hospital to test). I think that’s less likely to be “damage” and more likely to be “it’s still wearing off” or “I’m sorry, I just had minor surgery and you want me to take an IQ test?”. This made me throw out a lot of studies.

Few if any of the papers attempted to control for post-operative condition or pain med usage, which seems like an enormous oversight to me.

Tl;dr

My overall take home is that:

Little or nothing that necessitates surgery is good for cognition and that needs to be factored into assessments.

Sur

...
Against meta-ethical hedonism
by Joe Carlsmith — 2022-12-02T00:23:26.039Z

(Cross-posted from my website. Podcast version here, or search for "Joe Carlsmith Audio" on your podcast app.)

1. Introduction

Sometimes, people try to use meta-ethics to argue that pleasure and pain are the only things that matter. In particular, they say, we have a certain type of direct epistemic access to the goodness of pleasure and the badness of pain – access that we don’t have to other candidate values. So the value/disvalue of pleasure/pain are on solid meta-ethical footing in a way that other values are not.[1] Call this argument “meta-ethical hedonism.”[2]

I think this is a bad argument. This essay explains why. In brief:

Suppose you’re a non-naturalist about meta-ethics (that is, you think that the value of pleasure is something over and above pleasure itself, and irreducible to anything in the natural world). In that case, the argument fails to answer the basic epistemic objection to non-naturalism – namely, that it leaves us without the right type of epistemic access to the non-natural properties in question. In particular, positing a mysterious faculty of “direct acquaintance” with your phenomenal properties doesn’t help (even if such a faculty were otherwise legit), because phenomenal properties are natural properties.
Now suppose instead that you are naturalist about meta-ethics (that is, you think that normative properties are identical with some set of natural properties). In that case, your direct acquaintance (if you had it) with phenomenal properties – and thus, with a certain type of natural property – would still do no work in telling you which natural properties matter. Rather, to figure out that out, you need to do some combination of conceptual work, to understand how a concept like “goodness” picks out its natural referent, and empirical work, to check what that referent is. And direct acquaintance with the pleasantness of pleasure does neither.

Maybe this summary makes the issue sound complicated, but it really isn’t. Ultimately, pleasure is just another natural thing, and direct acquaintance, if it exists, just another way of detecting and comprehending a certain type of natural thing. Nothing unique has been offered that would bridge (or explain away) the divide between is and ought.[3] The basic game of normative epistemology remains unaltered.

Indeed, I suspect that the appeal of meta-ethical hedonism derives at leas...
Lumenators for very lazy British people
by shakeelh — 2022-12-02T00:18:36.876Z

My flat gets very little sunlight. In the British winter, it gets very very little sunlight.

I didn't like this and wanted to fix it. I read Chana's very good post, but was too lazy to do something like that, and also wanted something that took up less space and was less noticeable. In the comments, someone recommended a floodlight on eBay, but the item was no longer available.

So I did some googling, and found this: Viugreum 500W LED Floodlight, Daylight White 6000K. It's 50,000 lumens and £97.

You've got to attach the plug to the power cord yourself (buy this) and the power cord is very short so you want an extension lead (buy this) but once you've put in ~5 mins of work to do that, you're good to go.

Mine just arrived but my first impressions are pretty good. It's very, very bright. There's no flicker, and the colour rendering seems good enough. It's a pretty cold light, but that's not a bad thing for my main purpose (staying alert during the workday). I've got alternative lamps etc. I use at night.

The one big annoyance is that there's no switch on the floodlight, so you've got to turn it on and off from the mains.

I'm still figuring out the best placement. It's too bright to be in my direct line of vision. Currently it's on the floor near my desk, which seems to work well. It seems to get pretty hot, so you probably wouldn't want it on anything that might scorch or burn. Silver lining: you'll save money on your heating bill. Though I'm not looking forward to seeing my electricity bill next month.
A challenge for AGI organizations, and a challenge for readers
by Rob Bensinger — 2022-12-01T23:11:44.279Z

(Note: This post is a write-up by Rob of a point Eliezer wanted to broadcast. Nate helped with the editing, and endorses the post’s main points.)

Eliezer Yudkowsky and Nate Soares (my co-workers) want to broadcast strong support for OpenAI’s recent decision to release a blog post ("

Our approach to

alignment research

") that states their current plan as an organization.

Although Eliezer and Nate disagree with OpenAI's proposed approach — a variant of "use relatively unaligned AI to align AI" — they view it as very important that OpenAI has a plan and has said what it is.

We want to challenge Anthropic and DeepMind, the other major AGI organizations with a stated concern for existential risk, to do the same: come up with a plan (possibly a branching one, if there are crucial uncertainties you expect to resolve later), write it up in some form, and publicly announce that plan (with sensitive parts fuzzed out) as the organization's current alignment plan.

Currently, Eliezer’s impression is that neither Anthropic nor DeepMind has a secret plan that's better than OpenAI's, nor a secret plan that's worse than OpenAI's. His impression is that they don't have a plan at all.[1]

Having a plan is critically important for an AGI project, not because anyone should expect everything to play out as planned, but because plans force the project to concretely state their crucial assumptions in one place. This provides an opportunity to notice and address inconsistencies, and to notice updates to the plan (and fully propagate those updates to downstream beliefs, strategies, and policies) as new information comes in.

It's also healthy for the field to be able to debate plans and think about the big picture, and for orgs to be in some sense "competing" to have the most sane and reasonable plan.

We acknowledge that there are reasons organizations might want to be abstract about some steps in their plans — e.g., to avoid immunizing people to good-but-weird ideas, in a public document where it’s hard to fully explain and justify a chain of reasoning; or to avoid sharing capabilities insights, if parts of your plan depend on your inside-view model of how AGI works.

We’d be happy to see plans that fuzz out some details, but are still much more concrete than (e.g.) “figure out how to build AGI and expect this to go well because we'll be particularly conscientious about safe...
Playing with Aerial Photos
by jefftk — 2022-12-01T22:50:04.609Z

Flying in a window seat offers opportunities for serendipitous aerial
photography. You don't get to choose what is going to be within view
of the window, but when there's something interesting it can be fun to
try and get a good picture. There are a few I've taken that I like:

Our neighborhood, with Powderhouse Park in the lower right, Davis in
the middle right, Porter in the middle top, Trum Field in the lower
left, and the bike path running horizontally across the middle:

Did you know that after taking a boat up to the top of the Chesapeake
you can keep going, all the way to Philly? Here's the Chesapeake
& Delaware Canal:

Porter, looking East toward Kendall and Back Bay:

Downtown Boston:

Manhattan:

Singing Beach, Manchester by-the-sea:

Castle Island, right next to the East end of the container port::

More recently I've been playing around with a cheap telescope
attachment. Pictures are generally pretty blurry, but occasionally I
get something nice. Here's Atlantic City from about 70mi NW:

Comment via: facebook, mastodon
Take 1: We're not going to reverse-engineer the AI.
by Charlie Steiner — 2022-12-01T22:41:32.677Z

As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day for 25 days. Or until I run out of hot takes, which seems likely.

Any approach to building safe transformative AI, or even just auditing possibly-safe TAI, which relies on reverse-engineering neural networks into fine-grained pseudocode based on mechanistic understanding should keep its ambitions very modest.

This hot take is to some extent against ambitious "microscope AI," and to some extent against a more underlying set of intuitions about the form and purpose of interpretability research. (A somewhat related excellent background post is Neel's list of theories of impact for interpretability.) So I should start by explaining what those things are and why they might be appealing.

Webster's Dictionary defines microscope AI as "training systems to do complex tasks, then interpreting how they do it and doing it ourselves." Prima facie, this would help with transformative AI. Suppose we're building some AI that's going to have a lot of power over the world, but we're not sure if it's trustworthy - what if some of its cognition is about how to do things we don't want it to be doing? If we can do microscope AI, we can understand how our first AI is so clever, and build a second AI that's just as clever and that we're sure isn't doing things it shouldn't, like running a search for how best to deceive us.

Microscope-powered auditing is easier - if it's hard to assemble the second AI that does good things and not bad things, how about just checking that the first AI is trustworthy? To check an AI's trustworthiness in this microscope-AI-like framing of the issues, we might want to understand how its cognitive processes work in fine-grained detail, and check that none of those processes are doing bad stuff.

When I say I'm against this, I don't mean auditing is impossible. I mean that it's not going to happen by having humans understand how the AI works in fine-grained detail.

As an analogy, you can figure out how curve detectors work in InceptionV1. Not just in the sense that "oh yeah, that neuron is totally a curve detector," but in terms of how the whole thing works. It's yet more difficult to figure out that other neurons are not curve detectors - typically at this point we fall back on data-based methods like ablating those neurons and then tryi...
Re-Examining LayerNorm
by Eric Winsor — 2022-12-01T22:20:23.542Z

Please check out the colab notebook for interactive figures and more detailed technical explanations.

This post is part of the work done at

Conjecture

.

Special thanks to Sid Black, Dan Braun, Carlos Ramón Guevara, Beren Millidge, Chris Scammell, Lee Sharkey, and Lucas Teixeira for feedback on early drafts.

There's a lot of non-linearities floating around in neural networks these days, but one that often gets overlooked is LayerNorm. This is understandable because it's not "supposed" to be doing anything; it was originally introduced to stabilize training. Contemporary attitudes about LayerNorm's computational power range from "it's just normalizing a vector" to "it can do division apparently". And theories of mechanistic interpretability such as features as directions and polytopes are unhelpful, or even harmful, in understanding normalization's impact on a network's representations. After all, normalization doesn't alter the direction of vectors, but it still bends lines and planes (the boundaries of polytopes) out of shape. As it turns out, LayerNorm can be used as a general purpose activation function (you can solve MNIST with a LayerNorm MLP, for example). Concretely, it can do things like this:

and this:

We will explain what's going on in these animations later, but the point is that to develop a strong, principled theory of mechanistic interpretability, we need to grapple with this non-linearity.

In this interactive notebook, we study LayerNorm systematically using math and geometric intuition to characterize the ways in which it can manipulate data. We show that the core non-linearity of LayerNorm can be understood via simple geometric primitives. We explain how these basic primitives may perform semantic operations. For example, folding can be viewed as extracting extremal features (e.g. separating extreme temperatures from normal ones). We leverage these primitives to understand more complex low-dimensional classification tasks with multiple layers of non-linearities. The methods and intuition developed here extend to non-linearities beyond LayerNorm, and we plan to extend them in future work. Below is an interactive summary of the content of the notebook. (If you find the summary interesting, you really should check out the notebook: manipulating the figures helps build geometric intuition.)

Summary

The formula for LayerNorm is something messy like

Lay...
The LessWrong 2021 Review: Intellectual Circle Expansion
by Ruby — 2022-12-01T21:17:50.321Z

If you're already familiar with the LessWrong annual review, feel free to skip this opening and jump to What's new in 2021? or Unveiling the 2020 Annual Review books

The tradition continues! We will do for 2021 the same as we did for 2018, 2019, and 2020: approximately one year after the year is complete, we look back with the benefit of some distance and ask how good were the posts written then? After the hype has died down, do their claims still seem valid? Were they useful or interesting in a lasting way? Etc.

On LessWrong we're trying to make intellectual progress on problems that matter, but it can be hard to tell whether we're making progress. The Annual Review is the once-yearly Schelling time to reflect, debate, discuss and vote. It's a poke to say "remember that idea you and everyone else were obsessed with in 20xx? Did it continue to matter or was it just a fad?" It's also the time when we can determine which contributions most affected people in the long-term, and reward those authors and inspire others to follow suit. I think it's pretty great that when an author who invested hours into a post and greatly benefits the community, the community eventually recognizes and rewards them in some fashion.

The full technical details of how the Annual Review works is described in the final section of this post, but the core of it is the same as the past few years. There are three phases:

Preliminary Voting Phase (Dec 1- 14): Site members start identifying posts especially worthy of consideration in the review. Posts with 2 preliminary votes move into the Discussion Phase.
Discussion Phase (Dec 14 - Jan 11): Site members begin reviewing and debating posts in earnest. This discussion will inform members on how to cast their final votes. Posts that receive at least one written review move to the Final Voting phase.
Final Voting (Jan 11 to Jan 25): Site members allocate their vote budget (quadratic voting) amount nominated posts. The outcome of the final determines the Annual Review results.

For more of the philosophy of the Annual Review, see the previous announcement posts here and here and here.

Getting Started

If you are super eager to get started participating in the review, you can start preliminary voting on posts from the

All 2021 Posts page

, or the

View Your Past Upvotes

page. Note: only users with accounts registered before January 2021 are eligible to ...
The Plan - 2022 Update
by johnswentworth — 2022-12-01T20:43:50.516Z

So, how’s

The Plan

going?

Pretty well!

In last year’s writeup of The Plan, I gave “better than a 50/50 chance” that it would work before AGI kills us all (and my median AI timelines were around 10-15 years). That was an outside view, accounting for

planning fallacy

and the inevitable negative surprises. My inside view was faster - just based on extrapolating my gut feel of the rate of progress, I privately estimated that The Plan would take around 8 years. (Of those 8, I expected about 3 would be needed to nail down the core conceptual pieces of agent foundations, and the other 5 would be to cross the theory-practice gap. Of course those would be intermingled, though with the theory part probably somewhat more front-loaded.)

Over the past year, my current gut feel is that progress has been basically in line with the inside-view 8 year estimate (now down to 7, since a year has passed), and maybe even a little bit faster than that.

So, relative to my outside-view expectation that things always go worse than my gut expects, things are actually going somewhat better than expected! I’m overall somewhat more optimistic now, although the delta is pretty small. It’s only been a year, still lots of time for negative surprises to appear.

Any high-level changes to The Plan?

There have been two main high-level changes over the past year.

First: The Plan predicted that, sometime over the next 5 (now 4) years, the field of alignment would “go from a basically-preparadigmatic state, where we don’t even know what questions to ask or what tools to use to answer them, to a basically-paradigmatic state, where we have a general roadmap and toolset”. Over the past year, I tentatively think the general shape of that paradigm has become visible, as researchers converge from different directions towards a common set of subproblems.

Second: I’ve updated away from thinking about ambitious value learning as the primary alignment target. Ambitious value learning remains the main long-term target, but I’ve been convinced that e.g. corrigibility is worth paying attention to as a target for early superhuman AGI. Overall, I’ve updated from “just aim for ambitious value learning” to “empirically figure out what potential medium-term alignment targets (e.g. human values, corrigibility, Do What I Mean, human mimicry, etc) are naturally expressible in an AGI’s internal concept-language”...
Finding gliders in the game of life
by paulfchristiano — 2022-12-01T20:40:04.230Z

ARC’s current approach to ELK is to point to latent structure within a model by searching for the “reason” for particular correlations in the model’s output. In this post we’ll walk through a very simple example of using this approach to identify gliders in the game of life.

We’ll use the game of life as our example instead of real physics because it’s much simpler, but everything in the post would apply just as well to identifying “strawberry” within a model of quantum field theory. More importantly, we’re talking about identifying latent structures in physics because it’s very conceptually straightforward, but I think the same ideas apply to identifying latent structure within messier AI systems.

Setting: sensors in the game of life

The game of life is a cellular automaton where an infinite grid of cells evolves over time according to simple rules. If you haven’t encountered it before, you can see the rules at wikipedia and learn more at conwaylife.com.

A glider is a particular pattern of cells. If this pattern occurs in empty space and we simulate 4 steps of the rules, we end up with the same pattern shifted one square to the right and one square down.

Let’s imagine some scientists observing the game of life via a finite set of sensors. Each sensor is located at a cell, and at each timestep the sensor reports whether its cell is empty (“dead”) or full (“alive”). For simplicity, we’ll imagine just two sensors A and B which lie on a diagonal 25 cells apart. So in any episode our scientists will observe two strings of bits, one from each sensor.

(To be more realistic we could consider physically-implemented sensors, e.g. patterns of cells in the game of life which measure what is happening by interacting with the grid and then recording the information in a computer also built inside the game of life. But that adds a huge amount of complexity without changing any of our analysis, so for now we’ll just talk about these supernatural sensors.)

These scientists don’t know how the game of life works. All they see are the reports of the sensors. They observe that sensor B often fires 100 timesteps after sensor A, which they call an “A-B pattern.” Perhaps sensor A fires on 0.1% of timesteps, and sensor B fires on 0.1% of timesteps, but A-B patterns occur on 0.01% of timesteps instead of the 0.0001% you would have expected if the events were indepe...