trentmkelly/LessWrong-43k huggingface dataset:
Enlightenment Values in a Vulnerable World
by Maxwell Tabarrok — 2022-07-20T19:52:09.348Z
Link: https://maximumprogress.substack.com/p/enlightenment-values-in-a-vulnerable

Introduction:

The Vulnerable World Hypothesis: If technological development continues then a set of capabilities will at some point be attained that make the devastation of civilization extremely likely, unless civilization sufficiently exits the semianarchic default condition.

The Vulnerable World Hypothesis

(VWH) is an influential 2019 paper by philosopher Nick Bostrom. It begins with a metaphor for technological progress: An urn full of balls representing technologies of varying degrees of danger and reward. A white ball is a technology which powerfully increases human welfare, while a black ball is one which “by default destroys the civilization that invents it.” Bostrom stipulates that “the term ‘civilizational devastation’ in the VWH refers to any destructive event that is at least as bad as the death of 15 percent of the world population or a reduction of global GDP by > 50 percent lasting for more than a decade.” Given the dire consequences of such a technology, Bostrom argues for enlarged state capacity, especially in terms of global reach and surveillance, to prevent the devastating technology from being invented.

The VWH is a wet blanket thrown over Enlightenment values; values which are popular with many EAs and among thinkers associated with progress studies such as

David Deutsch

,

Steven Pinker

, and

Tyler Cowen

. These Enlightenment values can be summarized as: political liberty, technological progress, and political liberty ⇒ technological progress. Even if technology has a highly positive expected value on human welfare this can be easily outweighed by a small chance of catastrophic or existential risk. The value of political liberty is often tied to its promotion of technological progress. Large risks from technological progress would therefore confer large risks on political liberty. Bostrom highlights this connection but goes further. Not only is political liberty dangerous because of its facilitation of catastrophic technological risk, but strict political control is good (or at least better than you thought it was before) because it is necessary to prevent these risks. In response to a black ball technology Bostrom says that “It would be unacceptable if even a single state fails to put in place the machinery necessary for continuous surveillance and control of its citizens.” If Bostrom is right that even a small credence in the...
Countering arguments against working on AI safety
by Rauno Arike — 2022-07-20T18:23:26.574Z

This is a crosspost from my personal website.

One of the questions I’ve grappled with the most throughout the time I’ve been considering working on AI safety is whether we know enough about what a superintelligent system will look like to be able to do any useful research on aligning it. This concern has been expressed by several public intellectuals I respect. For instance, here’s an excerpt from Scott Aaronson’s superb essay „The Ghost in the Quantum Turing Machine“:

"For example, suppose we conclude—as many Singularitarians have—that the greatest problem facing humanity today is how to ensure that, when superhuman AIs are finally built, those AIs will be “friendly” to human concerns. The difficulty is: given our current ignorance about AI, how on earth should we act on that conclusion? Indeed, how could we have any confidence that whatever steps we did take wouldn’t backfire, and increase the probability of an unfriendly AI?"

And here’s Ben Garfinkel on the 80,000 Hours podcast:

"So just because something has some long run significance doesn’t necessarily mean that you can do much to affect the trajectory. So just a quick analogy; it’s quite clear that the industrial revolution was very important. /.../ At the same time though, if you’re living in 1750 or something and you’re trying to think, "How do I make the industrial revolution go well? How do I make the world better… let’s say in the year 2000, or even after that", knowing that industrialization is going to be very important… it’s not really clear what you do to make things go different in a foreseeably positive way."

Note that Aaronson probably doesn’t endorse his argument anymore, at least in the same form: he recently joined OpenAI and wrote in the post announcing his decision that he has become significantly more optimistic about our ability to attack this problem over time. However, as I’ve seen a similar argument thrown around many times elsewhere, I still feel the need to address it.

So, the argument that I’m trying to counter is "Okay, I buy that AI is possibly the biggest problem we’re facing, but so what? If I don’t have the slightest idea about how to tackle this problem, I might as well work on some other important thing – power-seeking AI is not the only existential risk in the world." I believe that even if one assumes we know very little about what a future AGI system will look like, there'...
A Short Intro to Humans
by Ben Amitay — 2022-07-20T15:28:45.669Z

A Preface to Intro to Humans

I used to understand humans. We were God's avatars on earth, sent to witness His Glory and enjoy His Grace - maybe up to a measurment error. Than God died, leaving some huge holes in my worldview. To address one of these holes I bought an Introduction to Psychology textbook, and read the "approaches to psychology" section. In this post I will try to lay out some main approaches to psychology and some relations between them, trying to understand them as different layers of the same map. My goal is not to inroduce any novel object-level ideas, but a high-level perspective.

Main Approaches to Human Psychology

Evolutionary Psychology

Attempts to understand human nature existed as "ethics" and "philosophy of mind" since forever, but the modern story began with Darwin's The Descent of Man, and Selection in Relation to Sex. As Chris Lerwill put it:

He points out continuity of features such as curiosity, imitation, attention, memory, imagination and reason, discusses the possible evolution of social structures involving sympathy, fidelity and courage, and concepts of beauty especially in relation to sexual selection. For many of his proposals he used evidence from cultures worldwide, both from his own experience and from his extensive correspondence with anyone that he felt might provide information beneficial to his developing ideas. The Descent of Man also made a novel contribution to psychology in that it was the first time that anybody had looked at humans as a naturalist observing a species of mammal or indeed animal (almost 100 years before Desmond Morris and The Naked Ape) and was able to step outside of our species and look at us as if as an alien naturalist.

And so, the seed of modern psychology was planted - with the understanding that humans are part of the natural world, and a potential subject for scientific investigation just like any other part. It was also the source of more specific priors, that heavily influenced the later schools:

The rerefance class: We are to be understood in the context of their place in the tree of life. Everything we discover about other animals/mammals/apes is potentially relevant for understanding humans. Everything that make humans unlike other animals/mammals/apes is apriori surprising, and worth extra scientific attention.
The generative model: We were created by a specific

...
How to Diversify Conceptual Alignment: the Model Behind Refine
by adamShimi — 2022-07-20T10:44:02.637Z

This post is part of the work done at Conjecture.

Tl;dr: We need far more conceptual AI alignment research approaches than we have now if we want to increase our chances to solve the alignment problem. However, the conceptual alignment field remains hard to access, and what feedback and mentorship there is focuses around few existing research directions rather than stimulating new ideas. This model lead to the creation of

Refine,

a research incubator for potential conceptual alignment researchers funded by the LTFF and hosted by Conjecture. Its goal is to help conceptual alignment research grow in both number and variety, through some minimal teaching and a lot of iteration and feedback on incubatees’ ideas. The first cohort has been selected, and will run from August to October 2022. In the bigger picture, Refine is an experiment within

Conjecture

to find ways of increasing the number of conceptual researchers and improve the rate at which the field is making

productive mistakes

.

The Problem: Not Enough Varied Conceptual Research

I believe that in order to solve the alignment problem, we need significantly more people attacking it from a lot different angles.

Why? First because none of the current approaches appears to yield a full solution. I expect many of them to be

productive mistakes

we can and should build on, but they don't appear sufficient, especially with shorter timelines.

In addition, the history of science teaches us that for many important discoveries, especially in difficult epistemic situations, the answers don't come from one lone genius seeing through the irrelevant details, but instead from bits of evidence revealed by many different takes and operationalizations[1] (possibly unified and compressed together at the end). And we should expect alignment to be hard based on

epistemological vigilance

.

So if we accept that we need more people tackling alignment in more varied ways, why are we falling short of that ideal? Note that I will focus here on conceptual researchers, as they are the source of most variations on the problem, and because they are so hard to come by.

I see three broad issues with getting more conceptual alignment researchers working on wildly different approaches:

(Built-in Ontological Commitments) Almost all current attempts to create more conceptual alignment researchers (

SERI MATS

, independent mentoring

...
What are the simplest questions in applied rationality where you don't know the answer to? 
by ChristianKl — 2022-07-20T09:53:01.600Z

In the podcast between Spencer Greenberg and Buck Shlegeris, Taking pleasure in being wrong, Buck says:

I think that when you are learning subjects something that you should really be taken your eye out for is the simplest question in the field that you don't know the answer to.

I think a lot of the time people try to learn physics or something and their approach is as quickly as possible to answer hard questions about complicated subjects. And I think that's what I thought was cool when I was younger. They delighted at questions that were at the limit of fanciness that they could possibly answer and it feels to me now that it is a lot more productive to seek out questions that are as simple sounding as possible while still being really hard to answer. Or that still demonstrate that there's something you don't understand about the subject.

[...]

It seems like we should be seeking out these most basic questions in the hope of finding holes in the foundation of our knowledge.

If we apply that approach to applied rationality, what questions do you have that seem to be simple but where you don't know the answer?
AI Safety Cheatsheet / Quick Reference
by Zohar Jackson — 2022-07-20T09:39:15.464Z
Link: https://github.com/Jakobovski/ai-safety-cheatsheet

I have created an AI safety cheatsheet which contains a compilation of the big AI safety ideas, problems and approaches to solutions.

https://github.com/Jakobovski/ai-safety-cheatsheet

In the process of learning about AI safety I found it useful to organize all the different ideas and topics and in one place that is easy to access and review.

I hope this cheatsheet provides value to the community and can serve as a quick reference.

Contributions are more than welcome, simply make a merge request and I will quickly approve it.
Getting Unstuck on Counterfactuals
by Chris_Leong — 2022-07-20T05:31:15.045Z

I've previously argued that counterfactuals only make sense from within a counterfactual perspective[1]. The problem then was that I didn't exactly know how to imply this insight because I had no idea how to navigate a circular epistemology. I concluded a) I should probably delve into the philosophical literature to discover how other people have handled this b) we could only evaluate approaches to navigate a circular epistemology from within a particular epistemological frame, so the circularity applies on a meta-level as well.

After writing that post, I got busy with my then web development job/interning at Nonlinear, ran the Sydney AI Safety Fellowship, then otherwise got busy with AI safety movement building, so I sadly hadn't been able to invest the time to make further progress.

However, today I had a call with Stephen Casper. I noticed that he likes to talk about thinking of someone as an "object" or as an "agent" which mirrors my distinction between "raw reality" and "augmented reality" as described here.

He also linked me to his post Dissolving Confusion around Functional Decision Theory. This post argues that:

Questions in decision theory are not questions about what choices you should make with some sort of unpredictable free will. They are questions about what type of source code you should be running.

How this plays out though is that he imagines an agent outside of the universe using CDT to determine what the source code of an agent inside the universe should look like and he argues that the internal agent should use FDT. Of course, this begs the question: if we've just concluded that CDT was a mistake, why would we trust its recommendation to use FDT? Isn't this argument self-defeating?

I believe that the answer becomes clear once we understand that this is part of a process of reflective equilibrium. Sure, the fact that we utilised a decision theory framework that we then rejected isn't ideal, but perhaps it's the best we can do given that we need to reason from somewhere; we can't reason from outside of a framework (the view from nowhere). So making this shift from CDT to FDT is at least a plausible way to handle circular reasoning.

I don't know if this is the best way to apply a circular epistemology, but I guess I now feel unstuck. Part of the problem is that before I was considering the question of how to handle circular epistemologies in ...
Pitfalls with Proofs
by scasper — 2022-07-19T22:21:08.002Z

This post distills

some

excellent

writing

on decision theory from Abram Demski about spurious proofs and Troll Bridge problems, and it has significant parallels with this

work

. It recently won a $250 prize in UC Berkeley Effective Altruism's AI safety distillation contest. The goal of this post is to make two difficult-to-understand topics, spurious proofs and troll bridge problems, as easy to understand as possible.

An Innocent Question

A highly intelligent system should have the ability to reason with logic. After all, being logical is a lot more useful than being illogical. Let’s design such a system – a robot named Rob.

The goal of a logical system is to prove things from axioms. Axioms are facts that are assumed to be true with no need for justification. They can either be facts about objects or inference rules. For example, “0 is a natural number” and “for any natural number, n, Successor(n) is also a natural number” are both axioms of a widely-studied logical system called

Peano Arithmetic

. Axioms are often applied to other axioms. For example, I can use the two aforementioned ones to prove that Successor(0), a.k.a. the number 1, is a natural number. We call such a combination of axioms that reveals a new fact a proof and the result a theorem.

The goal of using logic is to teach ourselves useful things by proving them. And toward this end, we sure hope that our logical system can’t prove contradictions. If so, it wouldn’t be useful at all! [1]

Now back to Rob. Suppose we want him to use some logical system by which he reasons about himself and the world. Exactly what logical axioms he uses aren’t important as long as they are sufficiently expressive.[2] As we are tinkering away in the process of designing him to be highly-intelligent, suppose that Rob asks us a simple question.

Answering this question will lead us through some strange dilemmas, but when we’re done, understanding the solution will teach an important lesson about instrumental rationality: making a hypothetical commitment to doing something that would hypothetically be insane isn’t just ok – it can be really useful. Ultimately, understanding this principle will help us to understand how even highly-advanced AI systems could subtly fail if they aren’t paradox-proof.

Obeying the Proof: What Could Go Wrong?

A lot.&...
A daily routine I do for my AI safety research work
by scasper — 2022-07-19T21:58:24.511Z

I’m a Ph.D student at MIT working in AI safety--mostly interpretability in deep networks. I thought I would very briefly share a list of links that I have bookmarked and go through every weekday. It usually takes me <15 minutes. Since I started doing this, I’ve been much more up to date and informed on current work.

First is

Twitter

. For a long time, I resisted using Twitter because it’s terrible. But after I started following almost exclusively AI safety people and orgs, I’ve found a lot of good papers and ideas.

Second, I check

LessWrong

and the

AI Alignment Forum

:)

Finally, I look over new titles on arXiv for

AI

,

ML

,

Computation and Language

, and

Computer Vision and Pattern Recognition

. More often than not, I find a new paper that I want to click on and check out.

And as a bonus, I’d of course also recommend

these

three

newsletters

.
Progress links and tweets, 2022-07-19
by jasoncrawford — 2022-07-19T20:50:58.154Z
Link: https://rootsofprogress.org/links-and-tweets-2022-07-19

Links

The first images from the JWST look amazing. See also this comparison with Hubble
Helion Energy is hiring. “Clean, continuous energy for 1 cent per kilowatt-hour”
Lant Pritchett on the difference between anti-poverty programs and poverty reduction. To paraphrase Rutherford, all poverty reduction is either economic growth or stamp collecting
Joel Mokyr reviews Koyama & Rubin’s How the World Became Rich
Potential for a vaccine against a broad category of SARS-like coronaviruses (in Science; see also the announcement from Wellcome Leap which funded this)

Threads

Many waste byproducts were developed in Britain during the Industrial Revolution (@VincentGeloso). See also: In capitalism they use every part of the animal
“Reclosers” on electrical utility poles (@HillhouseGrady)

Queries

Why is hating humanity acceptable? (@CineraVerinia)
Why doesn’t washing hands create hand-washing resistant bacteria? (@_brianpotter)

Tweets

Request for blog: the history of social change, and lessons for modern movements
Eradicating smallpox sometimes meant breaking the rules
India might already the most populous country in the world

Retweets

Terraform Industries can convert H2 and CO2 into natural gas (@TerraformIndies)
Montessori on the need to teach “the highest discoveries” of science (@mbateman)
There’s an Industrial Revolution board game set in 18th-c. Lancashire (@dkedrosky)
Applications are open for CFAR workshops in Prague this fall!
by John Steidley — 2022-07-19T18:29:19.172Z

tldr: CFAR’s running updated versions of our mainline rationality workshop this fall in the Prague area.

Apply here

.

===

CFAR workshops are back! After a nearly three-year hiatus during COVID, we're going to be running a series of four workshops this fall in the Czech Republic.

So what is a CFAR workshop again?

These workshops are four-and-a-half-day intensive retreats that are designed around the rationality techniques CFAR developed in its 10+ year history. These techniques vary: some we pulled straight from academic literature (looking at you TAPs), others we adapted from outside practices (Gendlin’s Focusing), while some we developed wholesale (hello Double Crux, Goal Factoring, and all that which is still unnamed). Our goal with this workshop is to create an environment in which participants are able to gain new insight into their minds and their decision-making processes.

Why would you come to a CFAR workshop?

Because you want to! That’s the most important part.

But maybe also you’ve got a sense of stuckness, and a new frame or tool might help you shift. Maybe it’s exactly the opposite of stuck; you’re moving and changing, and you want to be deliberate about how you move forward. Maybe you’re just intrigued and excited about this thing you’ve heard about from your friends. No guarantees from us on the result, but folks often walk away with insight, new fascinating friends, and some outstanding ideas. Most participants answer that they were glad they came (the only trick is figuring out how to select yourself into the correct reference class.)

We think that the workshop would be particularly valuable if you:

Are enthusiastic about rationality and you want to nerd out
Have an exploratory/playful/experimental mindset - you like to try things!
Are looking for an environment where you can have open and honest conversations about what you care about

A caveat, about this workshop...

It’s experimental. Which, if you’re familiar with CFAR’s history, is true of much of our workshops. If the material isn’t alive and real and practiced by our instructors, we don’t believe it will be for participants. And since it’s been a significant nearly-3-years since our last workshop, we and our beliefs have shifted a bit. It’ll still be fairly recognizable for those familiar with our applied rationality, but we want to set your expectations. The most no...
Sexual Abuse attitudes might be infohazardous
by Pseudonymous Otter — 2022-07-19T18:06:43.956Z

Content warning: sexual abuse, rape, arguably trivialising thereof.

When I was 11 an older boy used to pull me behind a door in one of the school corridors, trap me there, shove his hand down my pants, and touch my penis.

This happened about once a week for a couple of months, until I moved to a different school (for non-related reasons).

I knew it was taboo, but I didn't yet know about sex, and didn't really understand why. I never thought about it again till a few years later when he sent me a letter apologizing. My parents were extremely curious about what the letter was but I burnt it before they had a chance to read it.

I never told anyone about this until now, and even now I'm writing this pseudonymously.

Not because I'm ashamed or embarrassed. I'm not - I didn't do anything to be ashamed of. But because I'm fine.

I really am. I don't think this made any lasting impact to my life. I'm happy, well adjusted, married, successful etc. I just don't really think about what happened very much, but then again I rarely think about anything that happened to me when I was 11.

And yet I feel like society is telling me that I ought to be broken. That I've been sexual abused. That recovering from this will be a difficult painful process, probably requiring therapy. And I fear that if I tell someone that, they'll treat me like that's the case, and I might end up believing it about myself.

From a purely objective perspective, non-violent rape doesn't seem quite as bad as society makes it out to be.

It's obviously unpleasant and frightening, but we treat rape as one of the worst things that can possibly happen. We expect "rape victim" to become someone's whole identity. We expect them to need intensive therapy to put themselves back together.

And I'm sure for plenty of people that's true. But for plenty of others it's true only because we expect it of them. People fill the social role that's been made for them, even when that's not ideal for them.

I don't know what to do about this. How do we communicate that sexual abuse is really not ok, without making victims of it feel like it's worse than it actually is?

But at the very least, when you hear somebody's abuse story don't jump straight into treating them like a victim. Find out how they feel about it, and if they don't feel like it was that bad, there's really no need for you to make them feel like it was worse.