trentmkelly/LessWrong-43k huggingface dataset:
How To Observe Abstract Objects
by LoganStrohl — 2022-09-30T02:11:38.987Z

Opening Thoughts

What is this thing and what is the point of it?

I’m trying to build a branch of rationality that is about looking at ideas and problems “for real”, thinking about them “for real”, “as a whole person”, and “without all the bullshit in the way”. This is a mini workshop in that vein.

The exercises here are about original seeing. They’re meant to help you stretch and strengthen a couple kinds of perceptual muscles. Most of them are not much like “here is how to do the thing”; they’re more like “here’s some stuff that might conceivably lead to you independently figuring out what the thing is and how to do it”. So be ready to experiment. Be ready to modify my instructions according to your whims.

This endeavor will happen in three phases.

Phase One takes about twenty minutes[1] to complete, and stands alone pretty well. In it, you will directly observe a concrete object (something you can hold in your hands, like a carrot or a teacup). Phase Two takes about ten minutes, and leads you to summon an abstract object (something you can't hold in your hands, like scout mindset, the future, or whatever happens when you talk to your mother). Phase Three takes another ten minutes, and should be completed right after Phase Two. In Phase Three, you will directly observe an abstract object.

If you want to make a mini-workshop of this, block off an hour, and take breaks.

If all goes well, you'll leave with a greater ability to think about things originally. You'll be better at observing absolutely anything in ways that sometimes reveal new information you could not have uncovered through habit, convention, or rehearsal of your preconceptions.

Epistemic Status, History Of the Exercises, and Responsible Use

(Feel free to skip this part and go straight to Phase One, if the heading doesn’t interest you.)

I think the stuff in Phase One is moderately solid.

I first designed and ran something like it as a CFAR unit around 2018. I’ve been using it and tinkering with it since, in both pedagogical and personal contexts (n≈75 people), and it’s come to be an important part of how I guide people toward a more direct approach to thinking and solving problems. It’s only a first step into direct observation that’s probably not all that useful on its own, but I bet it’s a pretty good first step for most people.

However, a few people seem to have an overall cog...
Prioritizing Parental Sleep
by jefftk — 2022-09-30T02:10:01.576Z

Overall I've really enjoyed being a parent, but poor sleep has been
the hardest part. At times we've both been so tired that we weren't
able to think clearly about how to fix the problem, which can be very
tricky. We've figured this out more over time, however, and sleep has
improved with each successive kid (n=3). Here's the main things that
have worked for us, all in one place.

Note that kids and parents vary a lot: our three kids are different
from each other, and your kids likely even more so. I'm hoping that
many things on this list will be useful to many people, but it
wouldn't be surprising for several of them to be a poor fit for any
individual family. I've ordered them roughly down from the ones that I
think are most likely to work for anyone who tries them. Perhaps
unsurprisingly this is also roughly in the order of youngest to
oldest; my impression is kids diverge more over time as their
personalities come out.

Sleeping in multiple
rooms. If the baby waking up means only interrupting one parents
sleep that's about half as much sleep deprivation.

Blackout curtains. Small children
typically wake up with the sun, which means you're up to. If you can
keep their room solidly dark in the morning by blocking out the sun
you can shift their schedule to whenever is most convenient for you,
typically a later waking.

Using a Snoo
automated bassinet. It gently rocks the baby and shushes them back to
sleep when they wake. We used this with our youngest for the first
six months. It worked very well, and automated almost all of the
helping babies fall back asleep that I needed to do with the older
two. We weaned her off it much more gradually than they recommended,
with first weaning mode (stops rocking after the baby is asleep) and
then running it without rocking.

Sleep training. At some point they no longer need to feed as
often, but either don't know how to fall back asleep on their own at
the end of a sleep cycle or would enjoy getting a cuddle before going
back to sleep. I wrote about our
experience with our oldest, and how her naps got immediately better.
With our youngest, at 15m, we've recently dropped the last night feed,
and are in the process of teaching her that she needs to go back to
sleep on her own the whole night.

Letting kids know when they can get up. When kids are a bit
older, they'll often wake up near th

...
Bruce Wayne and the Cost of Inaction
by TurnTrout — 2022-09-30T00:19:47.335Z

My brother (DivineMango) and I (TurnTrout) wrote this first arc of a Batman fanfiction in the summer of 2020.

1: Don't Look Away

“The man’s starving. That doesn’t change if you look away, Bruce.”

Hand wrapped firmly around his son’s shoulder, Thomas Wayne directed his son’s flinching gaze towards a haggard face. The man looked crumpled, his beard long and unkempt, his eyes bagged and bloodshot. A tattered cardboard sign hung heavily from his neck:

HungryPTSDNo familyPlease help

Bruce was scared. At eight, he’d seen homeless people before, but he’d never connected the dots: this was a person, just out on the street.

Thomas walked towards the man, his designer leather shoes tapping out a confident pace against the pavement. In a smooth motion, he pulled out his billfold and kneeled to speak.

The man flinched. He seemed surprised anyone had even noticed he was there, and his eyes widened as he recognized the magnate of Wayne Enterprises. Ignoring the looks shot his way by the affluent passersby, Thomas spoke with the man. After about a minute, he pressed two crisp large bills into the man’s dirty hands; between the bills was the business card of one of Gotham’s better psychiatrists.

As the father and son resumed their stroll, Bruce looked up at his dad, his eyes still wide.

“What happened to him? Did he used to be a kid like me?”

Thomas considered his response as they walked down the boulevard and past the gleaming storefronts. To their right stretched a verdant park dotted by dark wooden benches, some of which encircled a duck pond. Together, they sat.

“His name is Oscar, Bruce. He’s seen terrible things. Sometimes, people, good people, can get hurt in ways that don’t heal. It’s not like skinning your knee—it’s like falling down a dark well, and no one comes to help you climb out.”

“That’s awful.”

“Yes, it is. It is awful.”

“I don’t want to… I shouldn’t get any more toys until we help these people.”

Thomas blinked. The musical conversation of a happy couple drifted by. The sun cast long and deep shadows across the park.

“I’m proud of you, Bruce. Never forget that feeling. But... that’s not right. If you only let yourself be happy when everyone else in the world is, you’ll never smile again. The world is big, and no man or woman can fix everything by themselves.”

“Why not?”, Bruce asked earnestly, legs swinging lazily under the bench, feet grazing the tips of the gra...
Distribution Shifts and The Importance of AI Safety
by Leon Lang — 2022-09-29T22:38:12.612Z

Preface

The following text is my submission for the

AI Safety Public Materials contest

. In it, I try to lay out the importance of AI Safety Research to people who, according to the winning conditions of the contest, have not yet engaged with AI Safety, Lesswrong, or effective altruism. As such, I’m eager to receive feedback on how accessible the text is to people outside of these communities. Additionally, I’d like to know where you think my framings might be misguided.

In my text, I focus on the distribution shift problem since it is an inner alignment problem, which increasingly feels like the most central challenge of AI Safety to me. However, the text does not focus on explicit inner optimizers or agency, even though this would strengthen the argument. The reason is that I think the distribution shift problem can already motivate existential risk well enough; the value of keeping the discussion focused therefore seems larger than the value of painting a complete picture. Nevertheless, in the appendix, I include a short overview of other safety concerns.

I thank Gabriela Jiang, Tom Lieberum, Shos Tekofsky, and Magdalena Wache for their useful comments and feedback on this text.

Introduction

This text covers the importance of technical AI Safety research. I give an explanation of why current machine learning paradigms might lead to the disempowerment of humanity in this century. With disempowerment of humanity, I mean any situation in which humans lose the ability to steer the future development of society, leading in the worst case to a loss of everything that we find valuable.

The argument is based on the distribution shift problem of contemporary machine learning, the increased dependence on AI and stronger distribution shifts that High-Level Machine Intelligence (HLMI) entails, and the likely development of HLMI in this century. I hope this text can motivate people to work on AI Safety research to prevent humanity's disempowerment.

Note that the distribution shift problem is only one of several safety problems that have been discussed over recent years. You can find a short overview in the appendix. A good starting point for learning more about the distribution shift problem specifically is the 2016 paper on

Concrete Problems in AI Safety

.

It will be useful to have basic knowledge about machine learning to appreciate this text. I include a...
Clarifying the Agent-Like Structure Problem
by johnswentworth — 2022-09-29T21:28:08.813Z

A few months ago, Thomas Kwa put together

a statement

of a potential “holy grail” of

Selection Theorems

and put a prize on it:

The ultimate goal of John Wentworth’s sequence

"Basic Foundations for Agent Models"

is to prove a

selection theorem

of the form:

Premise (as stated by John): “a system steers far-away parts of the world into a relatively-small chunk of their state space”
Desired conclusion: The system is very likely (probability approaching 1 with increasing model size / optimization power / whatever) consequentialist, in that it has an internal world-model and search process. Note that this is a structural rather than behavioral property.

John has not yet proved such a result and it would be a major advance in the selection theorems agenda. I also find it plausible that someone without specific context could do meaningful work here. As such, I’ll offer a $5000 bounty to anyone who finds a precise theorem statement and beats John to the full proof (or disproof + proof of a well-motivated weaker statement). This bounty will decrease to zero as the sequence is completed and over the next ~12 months. Partial contributions will be rewarded proportionally.

I’ll call this the Agent-Like Structure Problem for purposes of this post (though I’ll flag that other researchers might prefer a different formulation of the

agent-like structure problem

). For some background on why we might expect a theorem like this, see

What’s General-Purpose Search, And Why Might We Expect To See It In Trained ML Systems?

.

Since Thomas’ prize announcement, I’ve had a couple other people ask me about potential prize problems. Such people usually want me to judge submissions for their prize, which is a big ask - good alignment problems are usually not fully formalized, which means judging will inevitably involve explaining to people how their “solution” isn’t really solving the intended problem. That’s a lot of stressful work.

The goal of this post is to explain the Agent-Like Structure Problem (as I see it) in enough detail that I’d be willing-in-principle to judge submissions for a prize on the problem (see final section for terms and conditions). In particular, I’ll talk about various ways in which I expect people will solve the wrong problem.

What Do We Want To Show?

Here’s Thomas’ statement of the problem again:

Premise (as stated by John): “a system steers f

...
Where I currently disagree with Ryan Greenblatt’s version of the ELK approach
by So8res — 2022-09-29T21:18:44.402Z

Context: This post is my attempt to make sense of Ryan Greenblatt's research agenda, as of April 2022. I understand Ryan to be heavily inspired by Paul Christiano, and Paul left some comments on early versions of these notes.

Two separate things I was hoping to do, that I would have liked to factor into two separate writings, were (1) translating the parts of the agenda that I understand into a format that is comprehensible to me, and (2) distilling out conditional statements we might all agree on (some of us by rejecting the assumptions, others by accepting the conclusions). However, I never got around to that, and this has languished in my drafts folder too long, so I'm lowering my standards and putting it out there.

The process that generated this document is that Ryan and I bickered for a while, then I wrote up what I understood and shared it with Ryan, and we repeated this process a few times. I've omitted various intermediate drafts, on the grounds that sharing a bunch of intermediate positions that nobody endorses is confusing (moreso than seeing more of the process is enlightening), and on the grounds that if I try to do something better then what happens instead is that the post languishes in the drafts folder for half a year.

(Thanks to Ryan, Paul, and a variety of others for the conversations.)

Nate's model towards the end of the conversation

Ryan’s plan, as Nate currently understands it:

Assume AGI is going to be paradigmatic, in the sense of being found by something roughly like gradient descent tuning the parameters in some fixed architecture. (This is not intended to be an argument for paradigmaticity; attempting to align things in the current paradigm is a good general approach regardless (or so Nate understands Ryan to claim).)
Assume further that Earth's first AGIs will be trained according to a process of our choosing. (In particular, it needs to be the case that AGI developers can train for more-or-less any objective they want, without thereby sacrificing competitiveness. Note that this might require significant feats of reward-shaping.)
Assume further that most capability gains will be driven by something roughly like gradient descent. (Ryan has some hope that this plan would generalize to various other drivers of capability gains, but Nate hasn't understood any of the vague gestures towards those ideas, and as of April 2022, Ryan's plans

...
It matters when the first sharp left turn happens
by Adam Jermyn — 2022-09-29T20:12:17.484Z

Thanks to Evan Hubinger for comments on these ideas.

Introduction

A “sharp left turn” is a point where capabilities generalize beyond alignment. In a sharp left turn an AI becomes much more capable than aligned, and so starts to exploit flaws in its alignment.

This can look like Goodharting, where strong optimization pressure causes outer alignment failure because the base goal isn’t identical to what we want. Or it can look like deceptive alignment, where the model is aligned to proxy goals that aren’t identical to the base goal, but we don’t notice until the model is capable enough to make the proxies fail.

However they happen, sharp left turns are bad.

An important question, though, is: when will the first sharp left turn happen?

Timing

Consider three scenarios:

Weak: The sharp left turn happens before models are able to cause existential risks. For instance, maybe GPT-4 has a sharp left turn but isn’t able to reliably execute on its plans.
Strong: The sharp left turn happens once capabilities are very super-human and dangerous if misused. In this world AI’s are doing the bulk of the world’s scientific research by the time alignment unravels.
Human-level: The sharp left turn happens while capabilities are around human level, meaning both dangerous and not game-changingly helpful for research.

I think that the most dangerous of these is the human-level world.

In the “weak” scenario, we get to see the sharp left turn coming. We potentially get lots of experience with models looking aligned and then that alignment failing with increasing scale. In this world, we effectively get multiple shots at alignment because the first badly-misaligned models just aren’t that dangerous. We get to experiment with deceptively aligned models and get a grounded empirical understanding of why models become deceptively aligned and how we might stop that. In this world, alignment research gets a chance to become a paradigmatic scientific field before the models get too capable.

In the “strong” scenario, we get to do alignment research with super-human models that are still aligned. In this world, we don’t have to solve the technical challenge of alignment entirely on our own. Alignment research potentially gets much further along before we have to have an airtight solution and prevent sharp left turns.

Both of these seem like pretty hopeful worlds. By contrast, the world where sharp left tur...
Covid 9/29/22: The Jones Act Waver
by Zvi — 2022-09-29T18:20:02.103Z

The pandemic is over. Long live the Public Health Emergency.

HHS is set to extend the Covid-19 public health emergency by its standing deadline of Oct. 13.

If renewed on the deadline of Oct. 13, the next deadline would be Jan. 11, 2023.

This would make for the 11th renewal of the PHE since its declaration in January 2020 by former and current HHS secretaries Alex Azar and Xavier Becerra, respectively.

The renewal would occur as HHS plans to shift costs of Covid-19 vaccines and treatments to the commercial market, a process beginning this fall that is expected to take months.

Hospitals have advocated for the PHE’s extension.

There can be no clearer proof of the abuse of an emergency than extending it after publicly declaring that it is over.

Executive Summary

Covid is over but somehow also still an ‘emergency.’

Potential new treatment for Covid doing well in clinical trials.

Some people will continue to do way too much prevention for a long time.

Let’s run the numbers.

The Numbers

Kansas reported 476 deaths, of which I removed 400 as a presumed backlog.

I decided not to remove the 12,098 positive tests in Hawaii, although I am skeptical.

Prediction from last week: 330k cases (-9%) and 2,600 deaths (-8%).

Result: 316k cases (-11%) and 2,813 deaths (-2%).

Prediction for next week: 290k cases (-9%) and 2,750 deaths (-3%).

Mildly surprising not to see a larger decline in deaths.

I saw claims in my feeds earlier this week that we were seeing increases in cases in New York and other Northeastern states as new strains take over. The numbers don’t back that up and we continue to not see anything scary on that front.

Predictions

Deaths

Cases

BA.2.75.2

The contrast between how BA.2.75.2 and other recent variants are being covered and noticed, and how previous strains that grew at similar rates were being covered, is striking.

This certainly does have a lot of signs of big trouble (paper).

Thing is, the paper said that BA.2.75.2 had ‘profound escape’ yet it is being outcompeted by a different strain, BF.7, which is a substrain of BA.5.

Optimistic Chise thread on the study on Omicron boosters from last time.

Bloom Lab thread on another study, showing B-cells more ready for Omicron 6 months out than 1 month out from a shot. New information has updated me towards Omicron boosters being a modestly better ...
High-Impact Psychology (HIPsy): Piloting a Global Network
by Inga G. — 2022-09-29T18:16:02.498Z

Engaged with psychology or mental health? This is for you.

Impartial compassion. Rationality. Wellbeing. For a movement built on these values, EA likely underutilizes psychology professionals. Together with our supporters from the Global Priorities Institute, the Center for Effective Altruism, and the Happier Lives Institute, HIPsy aims to help people engaged with psychology or mental health maximize their impact.

Vision

We will follow in the footsteps of the EA Consulting Network, High-Impact Medicine, and High-Impact Athletes. Accordingly, the goal of HIPsy shall be to increase the likelihood of high-impact decisions, make collaboration and information processes more effective, and reduce the risk of value drift for people engaged in psychology or mental health.

Relevant resources shall be available, easy to access, and use:

up-to-date high-quality information,
career and work advice,
networking, and collaboration opportunities.

Psychological know-how shall be effectively acquired, shared, and used for EA. Psychology expertise is particularly needed in the fields of:

mental health and well-being, both within EA and globally,
community building, and outreach,
management, HR, and operations,
priorities research, applied research, effectiveness research,
x-risk-reduction and AI safety, e.g. awareness-building, and persuasion.

Summary: The goal for the next few months is to find out which of the many potential actions to prioritize, and how to address them most effectively. We will check what materials, events, and services are in-demand, and pilot some of them.

Join us and pre-sign-up

Here you can let us know which resources would be most valuable to you. We aim to provide the most wanted and most promising ones—very likely for free. Feel free to forward this to anyone who might be interested in benefitting from this:

5 min - for you if you have a background in psychology
5 min - for you if you are interested in, working in, or adjacent to the mental health sector
3 min - for everyone interested in increasing their impact using evidence-based psychology

You want to help more?

Let us know here

. If you'd like to collaborate, fund us, or if you have got any of the following skills, we want to hear from you: online content creation, running mentorship programs, hosting events, web-dev, design, community-building,...