trentmkelly/LessWrong-43k huggingface dataset:
I'm going to help you quit Facebook with some science
by Elo — 2018-04-12T03:09:30.324Z

Cross posted from http://bearlamp.com.au/im-going-to-help-you-quit-facebook-with-some-science/

I was a serial Facebook addict. I used to spend 2+ hours a day on Facebook, most days. Until I worked out how to change my mind.

Let's talk about the news feed. We all have this feeling that the news feed is drivel. Even curated, mine was still full of crud. Even super curated it was dull at best. Eventually I realised, something had to give.

As with many conflicts, indecision feels uncomfortable. Personally, I was super uncomfortable sitting in the cognitive dissonance of two conflicting beliefs:

Belief 1: Facebook is drivel and I want to spend less time on Facebook.

Belief 2: Facebook has good content from my friends that I want to keep up with.

There are three possible ways this can go. Either Facebook is in fact drivel and I will be happy to avoid it at any cost. Or Facebook has good in it and I'm staying around for the good stuff because I know it's worth it. Or Facebook is sometimes bad and sometimes good in some other complicated fashion, and I should check Facebook in some complicated intermittent fashion because of that...

This is how I worked out which belief was right.

You will need:

your news feed

pen and paper

5-10 minutes

Basic premise: Facebook has some good content and some bad content. But how much of each is ideal, acceptable or tolerable?

If there were 10 good posts for every 3 bad posts, I might be willing to accept that. Maybe I can take some rubbish with the good! I should visit more often. 10:3

If there was 1 good post to every bad post, I could still accept that. 1:1

If there was 1 good post for every 5 bad posts, maybe I could suffer that. After all, not everything is perfect. 1:5

But what if there was only 1 good post for every 10, 20, 30 bad posts? I don't think I'd be okay with that. 1:10

And if it was worse - 1 good for every 50 bad - that would be enough to leave the platform. There's no point digging for diamonds in a dung heap. I'd rather just read a book. 1:50

Think about the possible ratios and write down your pre-commitment. What ratios do you consider ideal, acceptable, and unacceptable? Don't worry about getting it perfect; you're an adult who can change their mind at any time. The purpose of writing down a ratio is to establish a baseline expectation before testing. I'm...
Two clarifications about "Strategic Background"
by Rob Bensinger — 2018-04-12T02:11:46.034Z

I've talked to a few people who misunderstood important parts of the "strategic background" discussion in https://intelligence.org/2017/12/01/miris-2017-fundraiser/#3.

First, at least two people thought the 1-8 numbered list was "MIRI's organizational plan" rather than "what we'd be least surprised to see happen in the world, conditional on good outcomes." MIRI is trying to de-confuse itself about step 8 and help put AGI developers in a better position in the future to select for AGI designs that are alignment-conducive, not trying to develop AGI.

Second, at least two other people misread "minimal aligned AGI" as "minimally aligned AGI", and thought MIRI was saying that developers should do the bare minimum of alignment work and then deploy immediately; or they saw that we were recommending building "systems with the bare minimum of capabilities for ending the acute risk period" and thought we were recommending this as an alternative to working really hard to achieve highly reliable and robust systems.

The MIRI view isn't "rather than making alignment your top priority and working really hard to over-engineer your system for safety, try to build a system with the bare minimum of capabilities". It's: "in addition to making alignment your top priority and working really hard to over-engineer your system for safety, also build the system to have the bare minimum of capabilities".

The idea isn't that you can get away with cutting corners on safety by keeping the system weak; per Eliezer's security mindset posts, a good plan should work (or fail safely) if the system ends up being a lot smarter than intended. Instead, the idea is that shooting for the bare minimum of capabilities adds a lot of value if your fundamentals are really good. Every additional capability a developer needs to align adds some extra difficulty and additional points of failure, so developers should target minimality in addition to alignment.
The Chromatic Number of the Plane is at Least 5 - Aubrey de Grey
by Scott Garrabrant — 2018-04-11T18:19:50.419Z
Link: https://arxiv.org/pdf/1804.02385.pdf

This is a long standing open problem in math. Many people learn about it as early as high school as an example of an open problem, because it is so easy to state:

How many colors does it take to color every point in the plane so that ever pair of points of distance exactly 1 from each other have different colors?

There are simple proofs that anyone here can understand that this number is between 4 and 7, and those were the only bounds we knew for a long time.

The lower bound was improved to 5 recently by Aubrey de Grey! The same Aubrey de Grey that you probably think of as the face of solving aging! He points at a concrete subset of 1567 points in the plane that cannot be colored with four colors.

This is super exciting. I think this is basically the main example of a simple open problem in math that we (used to) have no progress on.

I hope Aubrey de Grey negotiated the moral trade with the mathematicians successfully, and now that he solved one of their most beloved problems, they will start working on solving aging.
What I got out of 'Algorithms to Live By'
by rk — 2018-04-11T16:35:05.155Z

Sorry for the length. In a few paragraphs there's a reader's guide so you can skip around. I have tried to put the most valuable stuff up first. Consider only reading the introduction.

Starting from every moment, there are choices you could make. Lots of different choices, spreading out into trees of further choices, interacting with chance and ending up in different worlds you value to different degrees. As you think about which path to take, you learn more about what is likely on each branch. But as you gain more knowledge, you lose some opportunities: branches get left behind as you follow the track of waiting and thinking.

How are we supposed to figure out how to explore this space effectively? We need solutions that trade off integrating knowledge of the tree, future options and the cost of spending time thinking.

These are hard questions, and we don't have complete answers, but we might look to those who have studied similar problems. One field of study that has overlap with this conundrum is that of algorithms. We can look at algorithms as case studies in rationality.

Making decisions is hard, and computer science is partly the study of finding the best decision given time and space constraints -- and humans certainly face those constraints. We also face forgetfulness, impulsivity, weak mental maths ... But we at least face time and space constraints. Algorithms to Live By by Brian Christian & Tom Griffiths is an exploration of the applicability of algorithms from computer science to human decision problems.

Though the book is flawed, I have changed my behaviour in some ways because of it, and am considering changing others as well.

Here are the three changes I've made that have been most worthwhile so far:

When I first get a set of new options that is likely to stay stable into the future, I prioritise choosing a new option over repeating a good choice (from Explore / Exploit). Odds of around 2:1 / 66% confident that this is an improvement.

I schedule internet access (unless I need to be responsive), and have my phone on aeroplane mode most of the time (from Scheduling). Odds above 9:1 / 90% confidence that this has been an improvement, but I have doubts about its long term feasibility.

For many things (email, paper & computer files) I no longer worry about having a good organisational system. I am prepared to pay the search cost when I need som

...
The limits of corrigibility
by Stuart_Armstrong — 2018-04-10T10:49:11.579Z

I previously wrote a critique of Paul Christiano's Amplification/Distillation idea. Wei Dei pointed out that I had missed out a key part of Paul's idea, his use of corrigibility.

I'd been meaning to write about corrigibility for some time, so this provides a good excuse. For a general understanding of corrigibility, see here; for Paul's specific uses of it, I'm relying on this post and this one.

There have been some posts analysing whether corrigibility can be learnt, or whether corrigibility can nevertheless allow for terrible outcomes.

Summary of the claims of this post:

Strong corrigibility, outside of a few simple examples, needs to resolve the problem of figuring out human values. The more powerful and long-term the AI's decisions, the more important this is.
Once human values are figured out, corrigibility adds little to the mix.
Moderate versions of corrigibility can be useful along the way for building safe AGIs.

An example of corrigibility

Corrigibility is roughly about "being helpful to the user and keeping the user in control".

This seems rather intuitive, and examples are not hard to come by. Take the example of being a valet to an ageing billionaire. The billionaire wants to write his will, and decide how to split his money among his grandnephews. He asks you, his loyal valet, to bring him the family photo albums, which he will peruse while making his decision.

It's obvious that there are many way of manipulating him, if you wanted to. He won't have time to go through all the albums, so just by selecting some positive/negative photos for one nephew and putting them on top, you can skew his recollection and his choice.

That's clearly manipulative behaviour. Much better would be to select a representative cross-section of the photos, and give them to him. Or, even better, enquire about the criteria he wants to use for making his choices, and selecting a cross-section of representative and relevant photos for those criteria.

Complicating the example

Which billionaire?

This is where it gets complicated. Suppose your billionaire claims to only care about whether his grandnephews were kind and considerate. However, you happen to know that he also values flattery. What's more, a bit of gentle coaxing on your part will get him to admit that; once that's done, you can freely select photos based on eviden...
Trustworthy Computing
by Douglas_Reay — 2018-04-10T07:55:54.612Z

6. Trustworthy Computing

Summary of entire Series: An alternative approach to designing Friendly Artificial Intelligence computer systems.

Summary of this Article: How to use non-open-ended task programs to permit a computing environment in which AIs could trust each other enough to cooperate on ganging up on defectors sufficiently effectively that defecting becomes no longer the optimal strategy, and a stable society of AIs able to trust each other not to break agreements becomes possible.

Links to all the articles in the series:

Optimum number of single points of failure
Don't put all your eggs in one basket
Defect or Cooperate
Environments for killing AIs
The advantage of not being open-ended
Trustworthy Computing
Metamorphosis
Believable Promises

Links to the parts of this article

Recap
In the same league
Joining the posse
Group Programming
Showdown
Values Drift
The Human Touch

Recap

In an earlier part of this series ("Environments for killing AIs") we considered what humanity's computing environment might need to look like, if we wanted a posse of friendly AI's to be able to coerce other AI's into not trying to self-improve as fast as they can. We came up with some roles that might need to be fulfilled by a program or group of programs that all the AIs in the posse could trust to fulfill those roles and only those roles.

Gateway program - determines which programs may execute on the computing resources under its control (GlassNet, because it is transparent), according to some formula or schedule agreed between the members of the posse (and humanity). Think of it as granting cryptographic tokens that permit time-limited access via TOR to sandboxes running on anonymous machines around the world.

Watcher program - watches for signs of AI activity running on computing resources not accessed via the gateway (FogNet, because it isn't transparent).

Monitor programs - each monitor has read access to the inner data and workings of a single AI, and watches for lack of compliance with standards agreed between the posse and that AI.

Sheriff program - if the Watcher picks up signs of a new AI, or a Monitor signals that an existing AI has ceased to be compliant, the Gateway swings resources to a Sheriff that's tasked with investigating the situation, proposing a solution for the posse to confirm, then enacting the solution.

What would be need...
Announcing the Alignment Newsletter
by Rohin Shah — 2018-04-09T21:16:54.274Z

I've been writing weekly emails for the Center for Human-Compatible AI (CHAI) summarizing the content from the last week that's relevant to AI alignment. These have been useful enough that I'm now making them public! You can:

Sign up here
Read the first newsletter here
Bookmark the archive
ETA: Read the 5 emails I sent to CHAI (also on the archive)
ETA: There's also RSS!
Welcome to Moscow LessWrong group [Edit With Your Details]
by Alexander230 — 2018-04-09T18:29:10.851Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
The Alignment Newsletter #1: 04/09/18
by Rohin Shah — 2018-04-09T16:00:38.889Z

Highlights

Specification gaming examples in AI

(Victoria Krakovna): A list of examples of specification gaming, where an algorithm figures out a way to literally satisfy the given specification which does not match the designer's intent.

Should you read it? There were several examples I hadn't heard of before, which were pretty entertaining. Also, if you have any examples that aren't already listed, it would be great to send them via the form so that we can have a canonical list of specification gaming examples.

My take on agent foundations: formalizing metaphilosophical competence

(Alex Zhu): Argues that the point of Agent Foundations is to create conceptual clarity for fuzzy concepts that we can't formalize yet (such as logical uncertainty). We can then verify whether our ML algorithms have these desirable properties. It is decidedly not a goal to build a friendly AI using modules that Agent Foundations develop.

Should you read it? I don't know much about MIRI and Agent Foundations, but this made sense to me and felt like it clarified things for me.

Adversarial Attacks and Defences Competition

(Alexey Kurakin et al): This is a report on a competition held at NIPS 2017 for the best adversarial attacks and defences. It includes a summary of the field and then shows the results from the competition.

Should you read it? I'm not very familiar with the literature on adversarial examples and so I found this very useful as an overview of the field, especially since it talks about the advantages and disadvantages of different methods, which are hard to find by reading individual papers. The actual competition results are also quite interesting -- they find that the best attacks and defences are both quite successful on average, but have very bad worst-case performance (that is, the best defence is still very weak against at least one attack, and the best attack fails to attack at least one defence). Overall, this paints a bleak picture for defence, at least if the attacker has access to enough compute to actually try out different attack methods, and has a way of verifying whether the attacks succeed.

Technical AI alignment

Problems

Specification gaming examples in AI

(Victoria Krakovna): Summarized in the highlights!

Metaphilosophical competence can't be disentangled from alignment

(Alex Zhu): Would you be comfortable taking a single human...
Trust the (local) expert
by rk — 2018-04-09T10:22:13.133Z

Related: A Sketch of Good Communication, Argument, intuition & recursion

Your friend is wrong, but you look like the crackpot. They might be studying the PhD in climate science, but you're deferring to world-renowned climate expert, Prof. Knows-a-Lot! And yet everyone's looking at you weirdly ... You feel like Morgan that time they tried to argue with Lisa. Even though Lisa works for the IMF and Morgan had sort-of read an interview with Nobel-winning economist ...

I think it's likely that the friend with the PhD has the right of the matter, and the friend with the global expert in their corner has it wrong. I mostly think that because often, experts didn't say what we think they said. That is, we misunderstood them. And this is less likely to happen in a direct conversation.

It feels like we should be able to absorb expert opinions and shortcut to correctness on many matters. And fortunately we have access to the views of many top experts: whether books, lectures or editorials, the considered views of the elite are available to the masses. But what to do in a case like the vignette at the beginning of this post: we confront another, lesser expert who assures us we are wrong. Maybe not even wrong, but fundamentally confused! It feels like we can't just flip-flop on our views whenever we hear of a disagreeing expert. Why on earth should we prefer an expert that's close to us in space over one that's far but preeminent? A plausible approach is to stick with the higher status expert.

But then, as I alluded to above, we've also all seen the other side of that. And it doesn't seem like the one quoting the expert is implementing a truth-tracking strategy. A possibility is that we're just responding to status markers. But I think there are good reasons to be sceptical of expert-quoting as an epistemic strategy.

Ideally, we'd like to find out the consensus position. We might not be able to go toe-to-toe with an expert we meet on the road, but at least we can say "I grant that you understand the situation better than I do, but a poll of economists found that 80% disagree with what you're saying now. So I'll assume that's true, even though I can't see a flaw in your argument". Sometimes we might not be able to research consensus in the timeframe in which we need to act. More commonly, consensus is just not available. So we need to figure out what to do wit...
How do we change our minds? A meetup blueprint
by ChristianKl — 2018-04-09T07:58:36.277Z

Last Sunday, we had a LessWrong Meetup in Berlin in a new location and format with a lot of new people and it went great. We had 28 participants (32 if you count 4 kids that one person brought along).

The topic of the meetup was "How do we change our minds?" This was my second iteration of this format. I ran it before in Hamburg where most people who came to the meetup also didn't know each other.

We scheduled the meetup on meetup.com to officially begin at 13:00 with the first half hour being about arriving at the venue and chatting with other people. This setup makes it more likely that most participants will be in attendance at 13:30 and a setup I can generally recommend. Many people who attended the meetup heard about it through our existing mailing list or recommendations from friends.

Few people knew each other before the event started and little good conversations happened in the first half an hour. If you have more regulars in attendance, encourage them to start conversations with new people.

Fortunately, our event worked well and created common ground. By the end everybody was talking to each other and had great conversations.

Given that it worked so well, Janique Behman who was visiting Berlin asked me for the script for the event to be able to hold a similar event in another city. Given that it might be a valuable blueprint for other people who are thinking about what they could do on a Lesswrong meetup I decided to write it up as this post.

The general idea of the meetup is that all humans change their beliefs from time to time whether or not they already identify as aspiring rationalists or not. In the meetup every participant is tasked with writing a list of times when they changed their beliefs in the past year and afterwards we do exercises that build on that basis.

Instead of writing down the schedule as the event happened I write an idealized schedule based on what I learned:

13:30-13:40 Whole group: Introduction to the meetup and what LessWrong happens to be. Communicate the Chatham House rule for the event.

13:40-13:50 Whole group: Guided Meditation with theme of listening/incorporating new information (if you don't feel comfortable with leading meditations it's no problem to skip this step)

13:50-13:52 Whole group: Exercise explanation

13:52-14:02 Exercise 1: Solo work. Everybody takes 10 minutes to write down a list with ...
Critique my Model: The EV of AGI to Selfish Individuals
by ozziegooen — 2018-04-08T20:04:16.559Z

[Edit: Changes suggested in the comments make this model & its takeaways somewhat outdated (this was one desired outcome of posting it here!). Be sure to read the comments.]

I recently spent a while attempting to explain my views on the EV of AGI for selfish individuals. I attempted to write a more conventional blog post, but after a lot of thinking about it moved to a Guesstimate model, and after more thinking about it, realized that my initial views were quite incorrect. I've decided to simply present my model, along with several points I find interesting about it. I'm curious to hear what people here think of it, and what points are the most objectionable.

Model

Video walkthrough

Model Summary:

This model estimates the expected value of AGI outcomes to specific individuals with completely selfish values. I.E; if all you care about is your future happiness, how many QALYs would exist in expectation for you from scenarios where AGI occurs. For example, a simpler model could say that there's a 1% chance that an AGI happens, but if it does, you get 1000 QALYs from life extension, so the EV of AGI would be ~10 QALYs.

The model only calculates the EV for individuals in situations where an AGI singleton happens; it doesn't compare this to counterfactuals where an AGI does not happen or is negligible in importance.

The conclusion of this specific variant of the model is a 90% confidence interval of around -300 QALYs to 1600 QALYs. I think in general my confidence bounds should have been wider, but have found it quite useful in refining my thinking on this issue.

Thoughts & Updates:

1. I was surprised at how big of a deal hyperbolic discounting was. This turned out to be by far one of the most impactful variables. Originally I expected the resulting EV to be gigantic, but the discounting rate really changed the equation. In this model the discount rate would have to be less than 10^-13 to have less than a 20% effect on the resulting EV. This means that even if you have an incredibly low discount rate (10^-8 seems very low to me), you still need to consider it.

2. One reason why I made this model was to figure out if purely selfish people would have rational reasons to work on AI safety. At this point I'm not very sure. If the EV of it going well is +1k QALYs to a selfish person, then the EV of spending a year of hard work on making it tur...