trentmkelly/LessWrong-43k huggingface dataset:
The Achilles Heel Hypothesis for AI
by scasper — 2020-10-13T14:35:10.440Z

Pitfalls for AI Systems via Decision Theoretic Adversaries

This post accompanies a new

paper related to AI alignment

. A brief outline and informal discussion of the ideas is presented here, but of course, you should check out the paper for the full thing.

As progress in AI continues to advance at a rapid pace, it is important to know how advanced systems will make choices and in what ways they may fail. When thinking about the prospect of superintelligence, I think it’s all too easy and all too common to imagine that an ASI would be something which humans, by definition, can’t ever outsmart. But I don’t think we should take this for granted. Even if an AI system seems very intelligent -- potentially even superintelligent -- this doesn’t mean that it’s immune to making egregiously bad decisions when presented with adversarial situations. Thus the main insight of this paper:

The Achilles Heel hypothesis: Being a highly-successful goal-oriented agent does not imply a lack of decision theoretic weaknesses in adversarial situations. Highly intelligent systems can stably possess "Achilles Heels" which cause these vulnerabilities.

More precisely, I define an Achilles Heel as a delusion which is impairing (results in irrational choices in adversarial situations), subtle (doesn’t result in irrational choices in normal situations), implantable (able to be introduced) and stable (remaining in a system reliably over time).

In the paper, a total of 8 prime candidates Achilles Heels are considered alongside ways by which they could be exploited and implanted:

Corrigibility
Evidential decision theory
Causal decision theory
Updateful decision theory
Simulational belief
Sleeping beauty assumptions
Infinite temporal models
Aversion to the use of subjective priors

This was all inspired by thinking about how, since paradoxes can often stump humans, they might also fool certain AI systems in ways that we should anticipate. It surveys and augments work in decision theory involving dilemmas and paradoxes in context of this hypothesis and makes a handful of novel contributions involving implantation. My hope is that this will lead to insights on how to better model and build advanced AI. On one hand, Achilles Heels are a possible failure mode which we want to avoid, but on the other, they are an opportunity for building better models via adversarial training or the use of...
Have the lockdowns been worth it?
by Ben Pace — 2020-10-12T23:35:14.835Z

There’s been a lot of discussion about whether the pandemic lockdowns have been worth it. However, much of the reasoning that we’ve seen has been very motivated and un-nuanced in a way that for us has distorted a lot of the information.

So this is not a thread for taking a position on that. This is a thread for raising individual considerations that are relevant for thinking about the question “Have the pandemic lockdowns, in general, been worth it?”

Every answer to this thread should analyze a single belief that is relevant to whether pandemic lockdowns have been worth it, such as

"No lockdown would have led to 300,000 additional loss of QALYs in the UK"
"GDP $1b lower than the counterfactual in Germany"
"The West will develop a vaccine in 6 months"
“A year in lockdown is worth 70% of a normal year and 85% of a covid year without lockdown”
“The impetus to try remote work has led to changes in workplace norms worth $1 trillion over the next decade”

and provide relevant facts, data and information available about that factor.

Answers in this thread should not attempt to take a position on the overall question. Answers that take a position on the overall question will be deleted. This thread will live up to the virtue of

holding off on proposing solutions

.

(By ‘lockdown’ we refer to the thing that the US, UK and China have been doing, and what Sweden didn’t. There is naturally a lot of variation between countries, so this cannot have a canonical answer. If your consideration only applies to a small number of countries, that is fine.)
Police violence: The veil of darkness
by dynomight — 2020-10-12T21:32:33.808Z
Link: https://dyno-might.github.io/2020/10/12/police-violence-the-veil-of-darkness/

Measuring police bias using simple ratios doesn't work. You can never cleanly separate the impact of race from other factors associated with race.

Instead, I want augmented-reality goggles that make race invisible. Then we could run the following experiment:

Have half of police wear race-invisibility goggles for a year.
Have the other half wear non-invisibility goggles.
Look at the difference of the two groups.

The police with invisibility goggles would not have equal statistics with respect to race, because race is correlated with many things other than how people appear. However, since the only difference between the two groups is if police can see race, the difference reveals exactly the impact of police bias.

We can't do this experiment, but we can do kind of low-tech approximation. Instead of augmented reality goggles we use the geometry of the earth and sun. [continues here]
The Sun Room
by Jarred Filmer — 2020-10-12T21:08:11.507Z

I'm walking to work with a large coffee in hand. It is cold, and I am happy.

Because I am in the sun room.

I bring a task onto my mental desk. Considerations, insights, possibilities start spilling from it immediately with no effort required.

I seem to have micro thoughts within larger ones. Little jokes I make to myself, flashes of pleasant memories, ideas for tangential projects while the larger loops churn away.

I love to code here, though one must keep an eye for refactoring hubris when sitting in the sun room.

In this room I have the idea to start naming familiar states of consciousness like rooms of a house. The sun room, the doorless room, the infinity cupboard, the hole.

Sadly I've only found coffee for opening this door but daily use quickly builds tolerance in me.

So I'd very much like to know, has anyone else has found a key to the sun room?
A full explanation to Newcomb's paradox.
by solomon alon — 2020-10-12T16:48:46.416Z

Since I’ve read of Newcomb’s paradox on the sequences on less wrong, I’ve always thought there is something fundamentally wrong about timeless decision theory. I would end up coming up with an alternate explanation that seemed correct to me. I then searched it on Wikipedia and found that what I said was already said of course. But I’m still curios what the community thinks on the topic.
Newcomb’s paradox according to Wikipedia is as follows.

There is an infallible predictor, a player, and two boxes designated A and B. The player is given a choice between taking only box B, or taking both boxes A and B. The player knows the following:[4]

Box A is clear, and always contains a visible $1,000.
Box B is opaque, and its content has already been set by the predictor:

If the predictor has predicted the player will take both boxes A and B, then box B contains nothing.
If the predictor has predicted that the player will take only box B, then box B contains $1,000,000.

The player does not know what the predictor predicted or what box B contains while making the choice.
To be clear when we say the predictor is infallible we can either mean that the predictor has not made a single mistake, over hundreds of witnessed occurrences or that we are dealing with a actual infallible predictor either way what I am going to say is valid.
There are four possible ways in which the predictor is making his predictions.
1. He’s not actually making any predictions and is cheating. This form of cheating can be either changing what’s in the box after you choose. Using mind control on you or any way of cheating the problem. either way if this is the case you should obviously choose just box B since your decision changes what’s in the box.
2. The predictor can in some way see the future. In this case you should obviously choose just box B because your choice affects the past since the predictor knowing the future reverses the casualty of time and what you do now actually changes what was their in the past without any paradox.
3. The predictor is really good at figuring out personalities. This seems highly unlikely because all we would need would be for one person to come in with a dice or a coin, who has a personality of wanting to flip a coin to decide. However, they could be using tricks like only selecting people who they are extremely sure of. Either way, since the money is already in t...
How much to worry about the US election unrest?
by NoSignalNoNoise — 2020-10-12T03:39:55.105Z

Political tensions in the US are high. There have been a lot of mass protests in recent months, some of which have turned violent (often after police provocation). Donald Trump and Mike Pence have not to promise to respect the election result if they lose. Many on the left accuse them of plotting to steal the election and/or promote violence against their political opponents.

Frankly, I don't consider myself sufficiently well-calibrated on politics to know how much of this is bluster and how much is true cause for alarm. How worried should I be about the US plunging into dictatorship or widespread political violence?

Can people who understand this stuff better than I do please comment?
Everything I Know About Elite America I Learned From ‘Fresh Prince’ and ‘West Wing’
by Wei Dai — 2020-10-11T18:07:52.623Z
Link: https://www.nytimes.com/2020/10/10/opinion/television-culture.html

Rob Henderson is one of a few writers outside of the rationalist community that I follow obsessively, for his consistently high insight-to-length ratio, and his general writing style which appeals to me in some hard to articulate way. Here in a recent NY Times Opinion piece, he writes about social class, social mobility, and how (some) people form their values.

Whether upwards social mobility is a real value for many people (as it seems to be), and how to achieve it if it is, I think is an important topic that I seldomly see discussed around here, so this could be a good time to do that. Personally I seem to have little desire or motivation to join the "elite" or "upper class", and would even prefer my kids to not go to an elite university, which would foreclose much of the opportunity for them to do that as well. Is that wrong? If "it all adds up to normality", what might explain that "joining or staying in the elite class" is a real value? Or conversely, how can we be pretty sure that it's not? (To be clear, these are not rhetorical questions, and I'm expressing genuine moral/axiological uncertainty.)
All Lesswrong Posts by Yudkowsky in one .epub
by sone3d — 2020-10-11T13:17:20.520Z

This .epub has been generated using webpages-to-ebook. Thanks to j0rges user from Reddit for compiling the urls. The book contains all Eliezer Yudkowsky's posts from Lesswrong since the beginning. HPMOR posts have not been included.

An index with all the URLs included in the ebook can be found HERE.

>>DOWNLOAD EPUB

I have used the Lesswrong “corporative” map image for the cover. Let me know if I should change that.

Tools used: webpages-to-ebook, Greaterwrong, Calibre & Sigil.
Charisma is a Force Multiplier
by Davis_Kingsley — 2020-10-11T10:21:06.825Z

Sometimes, people talk about charisma as just a positive -- for instance, a lot of generic self-help or business advice assumes that charisma is generally good. On the other hand, you sometimes see people, often from "nerdier" communities, who say charisma is bad, an arbitary and unfair advantage, just a distraction from the facts, etc. [1]

I like to look at it from a somewhat different perspective. Military strategy sometimes discusses the concept of "force multipliers" - factors (often technological or strategic) that multiply the effectiveness of equipment or personnel. For example, precision weapons guidance is a force multiplier - if it takes dropping twenty normal bombs to hit a target but only one guided bomb, the guidance system could be considered a 20x multiplier on the effectiveness of your bombs. [2]

Perhaps the greatest force multiplier I am aware of for "normal situations" is charisma. Charismatic individuals are more likely to be effective and successful at a wide range of social tasks, more likely to influence broader planning towards their goals, more likely to elicit support from others, and so on. I would suggest that charisma is not another skill that's a useful addition to a broad range of skills; charisma can be a multiplier for all or almost all of an individual's other skills, at least insofar as the application of those skills involves working with others (and more skills do that than you might think)!

However, there is another important factor that should be taken into consideration. The fact that charisma can be such an important multiplier means that, if an individual's impact is negative, a more charismatic version of that individual will likely have a substantially more negative impact. [3]

Some of the most harmful results I've noticed in organizations and communities have arisen from very charismatic individuals who nevertheless pull the group in the wrong direction. I should note that I'm not saying that these individuals are necessarily malicious, evil, or similar - my view is that charisma is a skill like many others and not intrinsically good or bad. [4] However, one doesn't need to be malicious to be destructive - an appealing, persuasive person with the wrong take on what strategy to pursue can be very costly for an organization or community.

As such, I recommend exe...
Reviews of the book 'The Alignment Problem'
by Mati_Roy — 2020-10-11T07:41:14.841Z

Related: Goodreads – The Alignment Problem: Machine Learning and Human Values

The book 'The Alignment Problem' was published on October 6th, 2020.

Brian Christian posted on Facebook:

TODAY IS THE DAY!

After four years, a hundred interviews, 4,528 git commits, 1,057 references, 9,900 copyedits, and thirty hours in a recording studio, today THE ALIGNMENT PROBLEM finally enters the world.

The twin questions of ethics and safety in AI (more precisely, machine-learning) systems are, in my view, going to be the defining issues of the decade and perhaps the century. We are essentially (and in some cases literally) putting the world on a kind of autopilot, from our highways to our courthouses. There are many reasons to worry whether our fuzzy human intentions and sharp machine objectives, in fact, match.

Fortunately there is an incredibly vigorous movement rising to meet that challenge head-on, and it comprises some of the smartest and kindest people I've had the privilege to call colleagues and friends.

I've tried to tell the story of that harrowing and urgent challenge, our dynamic and exuberant response, the progress made and the problems remaining, and what it ultimately tells us about ourselves.

I hope you'll read, or listen, and can't wait to hear what you think.

https://brianchristian.org/the-alignment-problem/
Why Boston?
by jefftk — 2020-10-11T02:40:03.116Z

In college we had a set of signs we posted for common concepts. You
might point at the sign that said "lexical semantics" as convenient
way to indicate that a discussion had fallen into that particular
trap, and we had signs for many circumstances. The sign with the
largest impact, however, was "move to Boston". Twelve years later
it's worked surprisingly well, and ~85% of my friend group is now
here.

I'm also seeing a lot of discussion
among Bay Area friends about moving. With the pandemic, fire season,
and high rents, it's not surprising! I wanted to expand on a comment
I left about why I like Boston.

In no particular order:

Boston has a wide variety of industries, many of which are
quite strong: it's one of the top cities for biotech, medicine, and
education. This variety makes it easier to have friends in a range of
fields, there's interesting cross pollination, and there's less risk
that a bust in one industry will take the whole city down with
it.

There are good programming jobs. Many companies have offices
here to be more attractive to (especially MIT) students, and I feel
like I have a lot of options. Pay
is a bit below the Bay
Area, but not by much.

Good options if you don't
want to have a car. Many walkable areas, lots of bikes on the road
and decent bike infrastructure, and good public transit (for the
US).

Minimal natural disasters. Not an earthquake area, plenty of
rainfall, easy to avoid flood zones, no wildfires. Hurricanes happen,
though they pretty much are only an issue right on the water.
Occasional blizzards.

Pretty good governance, as illustrated by the pandemic
response. I live in Somerville, and we shut down more quickly and have
opened up more slowly than most of the US. Masks have been mandatory
since
late April. School is and has been fully remote, but they're
still distributing school lunches. They distributed laptops and
hotspots to any families that needed them. Non-pandemic governance
seems generally good as well.

Queer and poly friendly. For example, MA was the first state
to allow same-sex marriage (2003)
and Somerville was the first city in the country to issue polyamorous
domestic partnership (2020).

There are a lot of multi-unit houses and large old houses
that are suitable for use as group
houses, for people that like that.

Good schools. MA is typically ranked
top in the country for K-1

...
Rule of Equal and Opposite Advice & Slack
by tinyanon — 2020-10-10T21:22:57.321Z

References:

Slack:

https://thezvi.wordpress.com/2017/09/30/slack/

TL;DR Having a 20% buffer in your schedule to rest and handle anything urgent that pops up is the difference between a good life with time to experiment and being in a situation where one emergency breaks you and everything is on fire all the time.

Rule of Equal and Opposite Advice:

https://slatestarcodex.com/2014/03/24/should-you-reverse-any-advice-you-hear/

TL;DR The person who needs the advice to love their body is probably the person to end up on diet forums. The person who needs the advice to care about their health is probably going to end up on HAES forums. Each person needs to reverse the advice they naturally seek to get the advice that brings them to the healthy middle way rather than a self destructive extreme.

Main Post:

Calm > Anxiety; Slack > Distress:

The advice to seek out slack seemed to be the advice lesswrong needs. This forum is filled with people who worry about if they should be working 24/7 and donating all of their income. I think if your problem is on the anxiety spectrum it’s really important that you step back from trying to find ways to get everything done and try the opposite advice of seeking slack.

Joy > Depression; Eustress > Slack:

I think I’ve gone too far down the slack hole and need to reverse the advice again. This forum is also filled with smart people who need a challenge to feel fulfilled. As more slack has found its way into my life[1] I’ve been having trouble motivating myself to do the miniscule workload asked of me; whereas in the past I’ve handled herculean schedules and loved it.

As a result, I’ve been seeking more and more slack thinking “I can barely do this, I clearly need to try even less”. I need to reverse that. Of course, I don’t want to go back to not have a single unscheduled second in my day. The goal is to settle the pendulum at a nice healthy medium where I have an engaging number of balls to juggle but if I drop one I’ll still have time to pick it up before the next ball falls.

Conclusion:

If you’re feeling anxious, ask yourself where you can find more slack. If you’re feeling depressed, ask yourself where you can find things that excite you without making you anxious. Slack doesn’t necessarily mean having a lot of time where you do nothing. ...
Examples of positive-sum(ish) games?
by Nicholas / Heather Kross — 2020-10-10T21:09:04.349Z

How many positive-sum(ish) games, or PSIGs, exist?

For discussion, I'll be using a broader definition than normally used of positive-sum games:

Someone invents a magic free energy machine that outputs energy from nothing, breaking the laws of thermodynamics == PSIG
Two people cooperating to extract a resource from an otherwise barren asteroid == PSIG
Two people cooperating to steal a resource from other people != PSIG
Two people cooperating to play a zero-sum game against other people, each profiting more than they would have playing individually != PSIG
Anything where the negative externalities outweigh the positive externalities != PSIG

Hopefully this gives an idea of what kinds of examples I'm looking for.
Philosophy of Therapy
by DaystarEld — 2020-10-10T20:12:38.204Z
Link: http://daystareld.com/philosophy-of-therapy/

For a lot of people, therapy can be a confusing, mysterious thing of questionable value. Many have tried it when they were younger, and felt that at best it was only of minimal help, while for others it actually made things worse. In many cultures, therapy looks very different from how it’s practiced in the “western world,” and the concept of mental health itself is often treated with suspicion or dismissal. I’ve known many people who, even while not being skeptical, were still confused about what the purpose of therapy actually is, or what situations warrant seeking a therapist out.

In my practice as a therapist, I often reorient myself to the basic core of therapy, which to me is about helping people get unstuck. Sometimes the thing you’re stuck on is a recurring and disruptive emotional state, other times it’s some harmful interpersonal dynamic, and other times it's a pattern of behavior. Whatever the specifics, there is some aspect of the client’s life that is not going the way they would prefer, and the therapist’s job is to help them find a way to change that.

What the therapy provides also varies; good therapy can create space for honest expressions of emotion, provide new perspectives or insight, and offer new “tools" for the client to use in their lives, specific behaviors or mental motions that help move past the stuckness.

Those skeptical of therapy often wonder: can’t people just talk to their friends or family if they need emotional support? Aren’t there self-help books they can try? And of course they can, and should try those things! For many people, the majority of their difficulties do not require a therapist.

Which means therapy is for what’s left. Those things that seem truly intractable, the things that you feel stuck on, which other resources have failed to help resolve.

But I’d like to demystify therapy further, and better yet, I think by better understanding what therapy is meant to do and how, people can get some of the value that therapy can provide even without going to see a therapist.

Because while much of the change in therapy comes from the therapeutic relationship itself (which is why first finding a therapist that you feel comfortable with is half the battle), for a large portion of clients I’ve seen, even just changing the frame of the problems they experience, or changing the way they view themse...