trentmkelly/LessWrong-43k huggingface dataset:
Study on what makes people approve or condemn mind upload technology; references LW
by Kaj_Sotala — 2018-07-10T17:14:51.753Z
Link: https://www.nature.com/articles/s41599-018-0124-6

In four studies (including pilot) with a total of 952 participants, it was shown that biological and cultural cognitive factors help to determine how strongly people condemn mind upload. [...] The results showed that people who value purity norms and have higher sexual disgust sensitivity are more inclined to condemn mind upload. Furthermore, people who are anxious about death and condemn suicidal acts were more accepting of mind upload. Finally, higher science fiction literacy and/or hobbyism strongly predicted approval of mind upload. Several possible confounding factors were ruled out, including personality, values, individual tendencies towards rationality, and theory of mind capacities. [...]

We also found consistent age effects (older people condemn mind upload more than younger people), which together with our Science Fiction Hobbyism Scale imply that cultural factors, such as exposure to the ideas of consciousness transfer from young age through books, films and pop culture is also an important factor. In other words, familiarization with transhumanist themes together with biological components related to disgust and purity seem to play a significant role in determining whether people condemn or approve of mind upload technology. [...]

One of the more intriguing pieces of our results is the one that relates to Death Anxiety. In Study 2 we controlled for individual differences in death anxiety (Templer and Ruff, 1971) and suicide condemnation (Renberg and Jacobsson, 2003; Arnautovska and Grad, 2010), both of which—contrary to our initial expectations—predicted higher approval of mind upload technology. It is quite interesting that despite the scientist losing his actual physical body in the vignettes, his actions were not perceived as dying or as being suicidal. Throughout human history and cultures, there are myths about fountains of youth and stories about becoming immortal. It is noteworthy that these same motivations, dreams or intuitive existential and moral concerns surface in postmodern technological contexts. From this perspective, our findings link back to the perennial questions driving our civilizations; the results do seem reasonable since people who are anxious about death probably find the idea of extending life, or even transcending death, comforting (O’Connell, 2017; Kurzweil, 1999, 2004).

To our knowledge, we have now for the first tim

...
How to parent more predictably
by jefftk — 2018-07-10T15:18:33.660Z

(As with anything else I write about parenting, this is mostly just my observations with my own two kids and may not generalize as well as I think it does.)

A few weeks ago I wrote about why I think it's valuable for parents to be predictable. I was mostly describing an end state, though, and reasons why it's a good place to be, but what should you do if you're interested in being more consistent?

I think it breaks down into three pretty different skills. The first is only saying things you're comfortable fully standing behind. Many people especially when irritable, sleep deprived, or surprised, can quickly spit out a consequence that is harsher than is fair. Things like, "if you don't stop this minute it's no desserts for a week!" Then when the threat fails the consequence feels too extreme, and they don't follow through. If someone read my earlier post and decided they needed to stand by their quickly distributed threats I'd be pretty sad, and not expect this to make things better. Instead, any consequences need to be reasonable and proportionate, where you feel fair when you do have to enforce them (which, ideally, is rarely).

One thing that can be helpful is having a system of simple stock responses. The simplest is just a stern voice. If you reserve speaking firmly for rare circumstances, then saying something intensely and seriously can feel very significant.

Time outs can also work well: we use a system where we make it clear what needs to change ("stop shouting", "brush your teeth"), count to three, slowly and clearly, stopping if they do what we want, and if we get to three then it's time out. Serious things can be time out immediately. If you reliably use these, then not only does the kid understand how they work and what they mean but you don't have to come up with a novel appropriate response at a time when you're very stressed ("Do not bite me. That's time out.)

Even better, though, is avoiding commands and threats entirely. If they've been playing with a puzzle and they ask you to read to them, you can say you'll read once they clean up the puzzle (and depending on age maybe help them clean it up). They probably want you to read badly enough that they clean up, but if not that's ok too. You're using something they want you to do as leve...
Open Thread July 2018
by  — 2018-07-10T14:51:12.351Z

For those low-effort ideas that are worth a little attention but perhaps not a lot (yet).

(shouldn't a bot be posting these?)

Prompts for things to say:

What accomplishments are you celebrating from the last month?
What are you reading?
What subject has taken up most of your thoughts lately?
What is something your fellow rationalists don't understand, but should?
Three anchorings: number, attitude, and taste
by Stuart_Armstrong — 2018-07-10T14:21:24.904Z

I've shown that one cannot simultaneously deduce the preferences and rationality of an irrational agent - neither in theory, nor in practice.

To get around that problem, one needs to add extra assumptions - assumptions that cannot be deduced from observations. The approach that seems the most promising to me is to use the internal models that humans have - models of themselves and of other humans. Note that this approach violates algorithmic equivalence, since some of the internal structures of the human algorithm are relevant (this means it also violated extentionality and some version of functionalism).

Humans very often agree...

One thing that gives me some hope for that approach, is that humans very often agree with each other about when other humans, or themselves, are being irrational or reasonable. The agreement isn't perfect by any means - and we spend a lot of time debating the uncertain cases - but in many, many areas, most humans agree with each other, implying that most humans use similar internal models.

My favourite example of this is the anchoring bias, where, for example, a human is asked to consider the last two digits of their social security number, asked whether they would pay that amount for chocolates, then asked what price they would actually pay for those chocolates. The bias comes from the fact that the price they named was influenced by the two digits they named.

What's interesting is that almost everyone agrees that this is a bias; no-one argues the case that actually people really value pricing things close to numbers they've heard recently.

More preference, less bias

Now let's imagine a situation where there is no quoting of social security numbers, but there are two potential chocolate vendors, one of them of standard politeness, the second very rude. Without running the experiment, I'm confident that people will be willing to pay more in the first case than in the second.

This is very similar to the anchoring bias: the two situations differ by one detail, and the price is different.

Is this a bias? Here I expect more disagreement. Yes, technically, the rudeness of the vendor should be independent of the quality of the chocolates, but there are arguments that, in social situations, one should take these into account.

All on taste

Now let's consider a third "anchoring" situation, where the chocol...
The Dilemma of Worse Than Death Scenarios
by  — 2018-07-10T09:18:56.648Z

In this post I will write about what worse than death scenarios are, and how and why we should prevent them. I would recommend reading with caution if you are prone to worrying about this topic as this post contains ideas which may be very distressing.

A worse than death scenario can be defined as any scenario in which the observer would prefer to die than continue to live. A distinction is made between prefering to experience nothing and then resume living and prefering to experience nothing forever. For example, mos people would prefer to use general anaesthetic during a necessary operation, but if there was no anaesthetic, they would not choose to die. With the knowledge that the discomfort experienced during the operation is necessary to continue living, many would choose to go through with it (this can obviously vary depending on the operation and the observer).

As the observer would prefer to die in a worse than death scenario, one can assume that they would be willing to do anything to escape the scenario. Thus, it follows that we should do anything to prevent worse than death scenarios from occurring in the first place. It is our first priority. In my opinion positive scenarios cannot change this due to the following observation: there is no positive scenario in which you would do anything to *not* make it stop, at least in our current human form. I cannot think of any scenario which is so positive for the observer that they don't care if it is certain that it will kill them.

Worse than death scenarios vary in severity. The most basic example would be someone being kidnapped and tortured to death. If technology will allow immortality or ASI at some point, there are scenarios of much greater severity. The most extreme example would be an indefinite state of suffering comparable to the biblical Hell, perhaps caused by an ASI running simulations. Obviously preventing this has a higher priority than preventing scenarios of a lower severity.

Scenarios which could mean indefinite suffering:

1. ASI programmed to maximise suffering

2. Alien species with the goal of maximising suffering

3. We are in a simulation and some form of "hell" exists in it

4. ASI programmed to reflect the values of humanity, including religious hells

5. Unknown unknowns

Worse than death scenarios are highly neglected. This applies to risks of all severities. It seems very comm...
Newcomb's Problem In One Paragraph
by Chris_Leong — 2018-07-10T07:10:17.321Z

A TLDR of my recent post on Newcomb’s. If you have libertarian freewill, then Omega cannot predict your decisions based on the past with near perfect accuracy without backwards causation in which case you’d obviously one-box. Otherwise we can assume determinism. Then, “What decision should I make?” is misleading as it assumes the same “I” over different possible decisions. But given the precise state of the individual, only one decision is possible. Multiple possible decisions implies multiple slightly different versions of you, so the question becomes, “What (version, decision) pair achieves the best outcomes?”. Clearly this is whatever version gets the million. Further, “How can my decision affect the past?” becomes, “Why does Omega make a different prediction for a different version of me?", which isn't hard to answer.
Letting Go III: Unilateral or GTFO
by johnswentworth — 2018-07-10T06:26:34.411Z

Context: The first post gave a long list of examples of just-let-go design: a problem-solving approach/aesthetic based on giving up direct control over the system. The second post talked about giving up control as a visible signal of understanding. In this post, we get to main advantage of just-let-go design.

Suppose I’m lying in bed one day thinking about the problem of dishonest car-sellers. How can I get car-sellers to be honest about problems with their car?

I know! We need to pass a law which makes it a criminal act to lie about a car one is selling, so dishonest car-sellers get jail time.

Note the subtle shift from “I” to “we”. Even setting aside the likely ineffectiveness of such a law, passing laws is not within the space of things “I” can do over a weekend. “Laws we should pass” is great for facebook-filler, but not so great for practical ideas which I could personally implement.

What if we come at the problem from a minimum-control angle? We want to prevent car-seller dishonesty, while exerting as little control as possible.

Well, how about we disincentivize the seller from lying? That’s easy, we just need a contract which gives the seller some kind of liability for problems. This isn’t a complete solution yet, the details of that liability and its enforcement matter, but that’s tractable. The next question is implementation: having drafted such a contract, how do I get people to use it?

That’s a much easier problem than passing a law.

Just off the top of my head, I could create a startup called TrustyCar through which people buy and sell cars, and the main selling point is that the seller has some kind of liability for problems. Trustworthy sellers can obtain higher prices for their car by selling through TrustyCar, and buyers can obtain cars which they know are reliable. People have an incentive to start using it; nobody needs to force them. Indeed, I could charge them to use TrustyCar; their incentives still line up even if I collect a small fee.

In short: having found a minimum-control solution, I can implement it unilaterally. Passing laws is a thing “we” do, but brokering car sales is a thing “I” can do.

That’s a natural property of minimum-control solutions to problems, especially in the economic arena. The whole point is that we want a solution which doesn’t require controlling anyone else - therefore we can implement it ourselves. Who “we” is will ...
Context Windows: A Model of Unproductive Disagreement
by Zachary Jacobi — 2018-07-10T01:40:31.218Z

When you notice that you’re talking past someone, what does it look like? Are they ignoring all the implications of the topic at hand (“I know the invasion of Iraq is causing pain, but isn’t the most important question ‘did they have WMDs?'”)? Or are they avoiding the object-level point in favour of other considerations (“factory farmed animals might suffer, but before we consider whether that’s justified, shouldn’t we decide if we’re obligated to maximize the number of living creatures?”)?

I’m beginning to suspect that many confused, fruitless conversations are caused by differences in how people conceive of and process the truth. More, I think I have a model that explains why some people can productively disagree with anyone and everyone, while others are frustrated by even their closest friends.

The basics of this model come from a piece that

Jacob Falkovich wrote for Quillette

. He uses two categories, “contextualizers” and “decouplers”, to analyze

an incredibly unproductive debate

about race and IQ between Vox’s Ezra Klein and Dr. Sam Harris.

Klein is the contextualizer, a worldview that comes naturally to a political journalist. Contextualizers see ideas as embedded in a greater context. Questions of “who does this affect?”, “how is this rooted in society?”, and “what are the (group) identities of people pushing this idea?” are the bread and butter of contextualizers. One of the first things Klein says in his debate with Harris is:

Here is my view: I think you have a deep empathy for Charles Murray’s side of this conversation, because you see yourself in it [because you also feel attacked by “politically correct” criticism]. I don’t think you have as deep an empathy for the other side of this conversation. For the people being told once again that they are genetically and environmentally and at any rate immutably less intelligent and that our social policy should reflect that. I think part of the absence of that empathy is it doesn’t threaten you.

Klein is highlighting context that he thinks informs Harris’s views. He’s aware of the human tendency to like ideas that come from people we feel close to (

myside bias

) – or that put a stick in the eye of people we don’t like. Because of this, he’s suggesting that Harris believes Charles Murray’s points about race and IQ because they have a common enemy.

There are other characteristics of contextualizers. They ...
Fundamentals of Formalisation Level 5: Formal Proof
by philip_b — 2018-07-09T20:55:04.617Z

Followup to Fundamentals of Formalisation Level 4: Formal Semantics Basics. First post.

This is a new lesson of our online course on math formalizations required for AI safety research.

The big ideas:

Natural Deduction Proof System

To move to the next level you need to be able to:

Explain the difference between a formal system of proof and our informal notion of proof.

Derive proofs of logical formula in the system of natural deduction.

Why this is important:

Learning mathematical proof is hard, as you may have experienced in levels 2 and 3. By learning a formal system of proof you will make your own thoughts more rigorous, understand the smallest details that need to be covered to perform a proof, and build your intuition for informal proof.

For every lesson you have 2 options: do the whole thing, or skip to the questions and exercises in the end. The latter option is for people who suspect they already know the subject. It serves as a means of verifying or falsifying that hypothesis.
RAISE is looking for full-time content developers
by  — 2018-07-09T17:01:38.401Z

(crossposted from Facebook. Related to the reverse job)

(Tldr: RAISE is setting up a high-effort content development team in Blackpool, with free accommodation and fully covered living expenses. If you are interested, fill in the form.)

It has been a month or 2 since RAISE decided to professionalize. As a result, we have received roughly $10.000 in funding.

This helped us bolster our production pipeline. We have a few new videos on the verge of publication, and a series of guides to prerequisite studies (authored by Erik and Trent) is being published here on LessWrong.

We as a team are glad that things are moving forward, and we feel encouraged to take it to the next level.

How far can we stretch 10k? In the Bay area, such an amount would sustain a small team for a month. That’s obviously not enough to fulfill our ambition of distilling the whole of AI Safety.

It’s clear that as a movement, basing all of our operations in the most expensive cities of the world is a mistake. There is undoubtedly an advantage to being near loci of innovation and wealth, but that doesn’t always apply. RAISE and AI Safety camp were started in the Netherlands, where EA and LessWrong communities are only taking baby steps.

It’s clear that, in order to maximize efficiency, one should outsource as much work as possible to places where living is cheap. Much the same as what competition-driven industrial companies are already doing in today’s globalized world.

That’s why RAISE has decided that, come September, it’s content production team will move to the EA hotel in Blackpool.

The hotel, as its main product, provides free living cost to anyone that autonomously works on an EA project. I was lucky to already know Greg when he launched this excellent initiative. We have been in contact since, and I’m going to live (and possibly work) at the hotel until (at least) February. Johannes Heidecke will (most likely) join me end October. We are looking for several highly motivated AI Safety talents to join our team, preferably for 3 months or longer.

While there, we hope to set up a high-focus office environment. Removing as many distractions as possible, optimizing it for creating content of the highest quality.
And we will make an extraordinary effort, shutting up to do the impossible.

If you’re interested, apply here. If you want to explore options, feel free to email us, and we can chat about it open-...
Alignment Newsletter #14
by Rohin Shah — 2018-07-09T16:20:04.519Z

I've created a

public database

of almost all of the papers I've summarized in the Alignment Newsletter! Most of the entries will have all of the data I put in the emails.

Highlights

One-Shot Imitation from Watching Videos

(Tianhe Yu and Chelsea Finn): Can we get a robot to learn a task by watching a human do it? This is very different from standard imitation learning. First, we want to do it with a single demonstration, and second, we want to do it by watching a human -- that is, we're learning from a video of a human, not a trajectory where the robot actions are given to us. Well, first consider how we could do this if we have demonstrations from a teleoperated robot. In this case, we do actually have demonstrations in the form of trajectories, so normal imitation learning techniques (behavioral cloning in this case) work fine. We can then take this loss function and use it with

MAML

to learn from a large dataset of tasks and demonstrations how to perform a new task given a single demonstration. But this still requires the demonstration to be collected by teleoperating the robot. What if we want to learn from a video of a human demonstrating? They propose learning a loss function that given the human video provides a loss from which gradients can be calculated to update the policy. Note that at training time there are still teleoperation demonstrations, so the hard task of learning how to perform tasks is done then. At test time, the loss function inferred from the human video is primarily used to identify which objects to manipulate.

My opinion: This is cool, it actually works on a real robot, and it deals with the issue that a human and a robot have different action spaces.

Prerequisities: Some form of meta-learning (ideally

MAML

).

Capture the Flag: the emergence of complex cooperative agents

(Max Jaderberg, Wojciech M. Czarnecki, Iain Dunning et al): DeepMind has trained FTW (For The Win) agents that can play Quake III Arena Capture The Flag from raw pixels, given only the signal of whether they win or not. They identify three key ideas that enable this -- population based training (instead of self play), learning an internal reward function, and operating at two timescales (enabling better use of memory). Their ablation studies show that all of these are necessary, and in particular it even outperforms population based training with manual rew...