trentmkelly/LessWrong-43k huggingface dataset:
Defining the normal computer control problem
by  — 2017-04-25T23:49:53.708Z

There has been focus on controlling super intelligent artificial intelligence, however we currently can't even control our un-agenty computers without having to resort to formatting and other large scale interventions.

Solving the normal computer control problem might help us solve the super intelligence control problem or allow us to work towards safe intelligence augmentation.

We cannot currently keep our computers doing what we want easily. They can get infected with malware, compromised or they get updates that may be buggy. If you have sufficient expertise you can go in and fix the problem or wipe the system, but this is not ideal.

We do not have control our computers, without resorting to out of band manipulation.

Genes have found a way to control the reptilian brain and also the more powerful mammalian and human brains, somewhat, as discussed in the control problem has already been solved. And the system continues to run, our brains aren't reformatted when we get bad behaviours. Let us call this the normal computer control problem. We don't know how the genes do it, but humans tend to do what they would want (if they wanted things!) despite our flexibility. There is some control there.

This problem of control has been neglected by traditional AI as there it is not trying to solve a cognitive problem. It is not like solving chess or learning to recognize faces. It is not making anything powerful, it is just weeding out the bad programs.

Comparing the normal computer control and AI control problem

The AI control problem has been defined as asking the question

What prior precautions can the programmers take to successfully prevent the superintelligence from catastrophically misbehaving?

In this language the normal computer control problem can be defined as.

What type of automated system can we implement to stop a normal general purpose computer system misbehaving (and carry on with its good behaviour ) if it has a malign program in it.

To make the differences explicit:

The normal control problem assumes a system with multiple programs some good, some bad

The normal control problem assumes that there is no specific agency in the programs (especially not super-intelligent agency)

The normal control problem allows minor misbehaviour, but that it should not persist over time

These make the problem more amenable to study. These sorts...
Stupid Questions May 2017
by gilch — 2017-04-25T20:28:53.797Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.
Meetup : Melbourne Social Meetup: April
by Maelin — 2017-04-25T11:36:19.060Z

Discussion article for the meetup : Melbourne Social Meetup: April

WHEN:
28 April 2017 06:30:00PM (+1000)

WHERE:
The Bull and Bear Tavern, Flinders Lane, Melbourne

Come along for this month's Social Meetup, now on the exciting new schedule of the fourth Friday of the month!

FB event page: https://www.facebook.com/events/119088048648008/

Social meetups are informal get-togethers where we chat about topics of interest and have a couple of drinks together.

WHEN? Friday April 28, 18:30 until late. Don't worry about being on time, though - it's fine to rock up whenever.

WHERE? The Bull & Bear Tavern, on Flinders Lane (just a short walk from Flinders St Station).

FOOD? The B&B does reasonable traditional pub food and we usually share a few plates of wedges. For those who stay late, we often go for a late night meal at around 11pm

CONTACT? Any issues on the night, call or text Richard on 0421231789

Discussion article for the meetup : Melbourne Social Meetup: April
I Updated the List of Rationalist Blogs on the Wiki
by sapphire — 2017-04-25T10:26:51.234Z

I recently updated the list of rationalist community blogs. The new page is here: https://wiki.lesswrong.com/wiki/List_of_Blogs

Improvements:

-Tons of (active) blogs have been added

-All dead links have been removed

-Blogs which are currently inactive but somewhat likely to be revived have been moved to an inactive section. I included the date of their last post.

-Blogs which are officially closed or have not been updated in many years are now all in the "Gone but not forgetten" section

Downsides:

-Categorizing the blogs I added was hard, its unclear how well I did. By some standard most rationalist blogs should be in "general rationality"

-The blog descriptions could be improved (both for the blog-listings I added and the pre-existing listings)

-I don't know the names of the authors of Several blogs I added.

I am posting this here because I think the article is of general interest to rationalists. In addition the page could use some more polish and attention. I also think it might be interesting to think about improving the lesswrong wiki. Several pages could use an update. However this update took a considerable amount of time. So I understand why many wiki pages are not up to date. How can we make it easier and more rewarding to work on the wiki?
The 2017 Effective Altruism Survey - Please Take!
by Peter Wildeford — 2017-04-24T21:08:54.258Z

This year, the EA Survey volunteer team is proud to announce the launch of the 2017 Effective Altruism Survey.

-

PLEASE TAKE THIS SURVEY NOW! :)

If you're short on time and you've taken the survey in prior years, you can take an abridged donations-only version of the survey here.

If you want to share the survey with others, please use this fancy share link with referral tracking: http://bit.ly/2q8iy2m

-

What is this?

This is the third survey we've done, coming hot off the heels of the 2015 EA Survey (see results and analysis) and the 2014 EA Survey. (We apologize that we didn't get a 2016 Survey together... it's hard to be an all volunteer team!)

We hope this survey will produce very useful data on the growth and changing attitudes of the EA Community. In addition to capturing a snapshot of what EA looks like now, we also intend to do longitudinal analysis to see how our snapshot has been changing.

We're also using this as a way to build up the online EA community, such as featuring people on a global map of EAs and with a list of EA Profiles. This way more people can learn about the EA community. We will ask you in the survey if you would like to join us, but you do not have to opt-in and you will be opted-out by default.

Who should take this survey?

Anyone who is reading this should take this survey, even if you don't identify as an "effective altruist".

How does the survey work?

All questions are optional (apart from one important question to verify that your answers should be counted). Most are multiple choice and the survey takes around 10-30 minutes. We have included spaces for extra comments if there is some extra detail you would like to add (these are strictly optional).

At the end of the survey there is an 'Extra Credit' section with some more informal questions and opportunities for comment - definitely feel free to skip these questions.

Results will be shared anonymously unless you give your explicit permission otherwise.

Who is behind this?

The EA Survey is a all-volunteer community project run through .impact, which is soon changing it's name to "Rethink Charity". The results will not belong to any one person or organization.
Open thread, Apr. 24 - Apr. 30, 2017
by gilch — 2017-04-24T19:43:36.697Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
Unstaging Developmental Psychology
by Gordon Seidoh Worley — 2017-04-24T03:39:14.808Z
Link: https://mapandterritory.org/unstaging-developmental-psychology-fcdc5f9ba894

It’s no secret that I think there is some there there when it comes to developmental psychology. However, I must admit theories of psychological development have some weaknesses. One issue we run into right away is that they sound elitist, and for many that’s enough to reject them in the same way some people reject evolutionary psychology and human biodiversity as racist. Sure, developmental psychology can be abused to support status grabs and enforce social hierarchies, and I have no doubt some people intentionally try to do this, but this is not itself evidence against the ideas, just cautions against how we use the insights we draw from them.

The other major problem is that developmental psychology isn’t a category of strictly scientific theories. One of Kegan’s key insights was that developmental psychology is based on phenomenological methods and capta rather than scientific methods and data. This presents a problem when you try to evaluate developmental psychology through the lens of science as Sarah Constantine recently noted:

Overall, the experimental evidence that distinct, cumulative stages of human development exist is rather weak. The strongest evidence is for Kohlberg’s stages, and these (like all the other stages considered) are limited by the fact that they are measures of how people talk about moral decision-making, rather than what they decide in practice.

Higher stages correlate with positive results in many cases: people at higher Kohlberg stages are less likely to be criminals or delinquents, positive psychological strengths like self-esteem correlate with the Eriksonian ego strengths, and leadership development measures correlate with Kegan stage. This is evidence that developmental stages do often correspond to real psychological strengths or skills with external validity. We just don’t generally have strong reason to believe that they progress in a developmental fashion.

And given that phenomenological methods are not well developed because there are few folks trying to rigorously apply them relative to the way folks apply scientific methods, it’s reasonable to reject the notion that we should trust results based on phenomenology. Developmental psychology might be pointing at real phenomena, but phenomenological theories of developmental psychology are going to fail to meet standards of scie...
Help Us Find Your Blog (and others)
by Zvi — 2017-04-23T11:53:11.000Z

Previously: Against Facebook – Call to Action, Against Facebook – Details

Note (5/2): Due to my need to prepare for Magic: The Gathering’s Pro Tour: Amonkhet, updates to this blog and progress exploring new blogs will be slow for the next two weeks. I will still get to the full list in time, and anticipate finishing in May.

Inspired by posts by and conversations with Alyssa Vance, and the desire to do some tangible things beyond laying out my case, this post will attempt to start fixing the biggest problem that personal blogs have: by default, no one knows about them.

Ideally, this problem is addressed gradually over time, as those who do see such blogs post links to their best posts, and those who follow those links then discover the blogs, resulting in growth. Also helpful is the blogroll, which allows those who like one blog to discover other similar blogs that the author recommends. This is not the worst system, but it is not great either, and right now I am not even doing my part in having a proper blogroll.

I can at least fix that, and while doing so, give those who have blogs I do not know about a chance to find me. So here is the plan:

If you have a blog, comment here with your name, the name and concept of your blog, and a link to either your blog and/or what you consider your best post. I will then read at least one of your posts (at least a few thousand words, unless it is unreadable), leave one comment, and consider adding you to my RSS feed and blogroll.

If you don’t have a blog, but you want to start one, this post will still be around, and I get an email whenever anyone comments. Email me when you feel you have hit your groove and have a few good posts. If we are personal friends, one post is enough to get you on my RSS.

Whether or not you have a blog, you are also encouraged to name and link to one to three additional blogs that you feel are great that no one has mentioned yet, and I will check them out.

If something is R-rated (as in actually inappropriate for kids, not just dropping an occasional f-bomb) please say so. If something is X-rated, do not link to it here, as this is neither the time nor the place.

Note that for now I have turned off approval-first comment moderation for this blog. Let’s make sure I do not regret that!

To avoid getting in too far over my head: These commitments are good for the first 100 peopl...
Against Facebook: Comparison to Alternatives and Call to Action
by Zvi — 2017-04-22T16:44:15.000Z

Previously: Against Facebook: Details

Take Action: Help Us Find Your Blog (and others)

Epistemic Status: Shouting from the rooftops. For further details, see previous post.

This post is my recommendations for how to communicate online. If you need details and/or detailed justifications of my view of Facebook’s awfulness, check out Against Facebook: Details. I recommend reading it if either the details would interest you for their own sake, or you do not understand what I mean when I say that Facebook is out to get us.

I consider the non-obvious goals of a unified system to be

1: Minimize check-in requirements

When you feel the need to constantly check something or risk missing important things, that is very bad. You should minimize the number of places you need to check in, and the cost to checking in at those locations. You do not want that Skinner Box addictive drive to constantly hit refresh, but if you need to have it, have it in one place where you know right away if there is indeed something new and what it is. When Facebook is the source of important information and interaction, it adds another check-in point without taking away the need for others.

2: Know what you are responsible for seeing, and what others are responsible for having seen

There needs to be agreement that communication in some forms means you are responsible for seeing that information within a reasonable time frame, and equally important, you need agreement that communication in other forms does not carry this same obligation. Facebook operates in a grey area where people assume you have seen anything important, sometimes (in my personal experience) even if they have been explicitly told multiple times you never look at Facebook at all, but there is a real danger that any given post was never in your News Feed at all, let alone seen. Facebook needs to be in the second category, of things that no one is assumed to have seen unless they explicitly tell you (via a like, a response, or otherwise).

3: You need to be able to see everything you want to see, and know you have seen it, with a minimum of stuff you do not want

Advertisements are a negative, but so is being forced to see stuff you do not want along with stuff you do want, and having to sort through all of that. Ideally low-quality stuff is there when you want it but not mixed in too much with high-quality. You need a reasonable record o...
Against Facebook
by Zvi — 2017-04-22T16:43:35.000Z

Leads to: Against Facebook: Comparison to Alternatives and Call to Action, Help Us Find Your Blog (and others)

Note: WordPress seems to be eating line breaks. I hope I have them all fixed at this point.

Epistemic Status: Eliezer Yudkowsky writing the sequences. They sentenced me to twenty years of boredom. Galileo. This army. Chris Christie to Marco Rubio at the debate. OF COURSE! A woman scorned. For great justice. The Fire of a Thousand Suns. Expelling the moneylenders from the Temple. My Name is Susan Ivanova and/or Inigo Montoyo. You killed my father. Prepare to die. Indeed. It’s a trap. Tomfidence. I swear on my honor. End this. I know Kung Fu. Buckle up, Rupert. May the Gods strike me down to Bayes Hell. Compass Rose. A Lannister paying his debts. The line must be drawn here. This far, no farther. They may take our lives, but they will never take our freedom. Those who oppose me likely belong to the other political party. Ball don’t lie. Because someone has to, and no one else will. I’ve always wanted to slay a dragon. Persona!

This post is divided into sections:

1. A model breaking down how Facebook actually works.

2. An experiment with my News Feed.

3. Living with the Algorithm.

4. See First, Facebook’s most friendly feature.

5. Facebook is an evil monopolistic pariah Moloch.

6. Facebook is bad for you and Facebook is ruining your life.

7. Facebook is destroying discourse and the public record.

8. Facebook is out to get you.

A second shorter post will then lay out what I believe is the right allocation of online communication. Some readers will want to skip ahead to that one, and I will understand.

I felt I had to document my explorations and lay out my case, but I trust that most of you already know Facebook is terrible and don’t need to read 7000 words explaining why. If that is you, skip to the comparison to alternatives or the call to action at the end. I won’t blame you.

A Model Breaking Down How Facebook Actually Works

Facebook can be divided into its component features. Some of these features add value to the world. I will start with those, because they form the foundation of the trap. These are the friendly parts of the system, that are your friends. They are not out to get you. If the rest of the system was also not out to get us, or we had it under control, I would use the good and mixed parts more.

The good:

Contact Information

Facebook’s best reason to exi...
The AI Alignment Problem Has Already Been Solved(?) Once
by SquirrelInHell — 2017-04-22T13:24:38.978Z
Link: http://squirrelinhell.blogspot.com/2017/04/the-ai-alignment-problem-has-already.html

Hat tip: Owen posted about trying to one-man the AI control problem in 1 hour. What the heck, why not? In the worst case, it's a good exercise. But I might actually have come across something useful.

第一

I will try to sell you on an idea that might prima facie appear to be quirky and maybe not that interesting. However, if you keep staring at it, you might find that it reaches into the structure of the world quite deeply. Then the idea will seem obvious, and gain potential to take your thoughts in new exciting directions.

My presentation of the idea, and many of the insinuations and conclusions I draw from it, are likely flawed. But one thing I can tell for sure: there is stuff to be found here. I encourage you to use your own brain, and mine the idea for what it's worth.

To start off, I want you to imagine two situations.

Situation one: you are a human trying to make yourself go to the gym. However, you are procrastinating, which means that you never acually go there, even though you know it's good for you, and caring about your health will extend your lifespan. You become frustrated with this sitation, and so you sign up for a training program that starts in two weeks, that will require you to go to the gym three times per week. You pay in advance, to make sure the sunk cost fallacy will prevent you from weaseling out of it. It's now 99% certain that you will go to the gym. Yay! Your goal is achieved.

Situation two: you are a benign superintelligent AI under control of humans on planet Earth. You try your best to ensure a good future for humans, but their cognitive biases, short-sightedness and tendency to veto all your actions make it really hard. You become frustrated with this sitation, and you decide to not tell them about a huge asteroid that is going to collide with Earth in a few months. You prepare technology that could stop the asteroid, but wait with it until the last moment so that the humans have no time to inspect it, and can only choose between certain death or letting you out of the box. It's now 99% certain that you will be released from human control. Yay! Your goal is achieved.

第二

Are you getting it yet?

Now consider this: your cerebral cortex evolved as an extension of the older "monkey brain", probably to handle social and strategic issues that were too complex for the old mechanisms to deal with. It evolved to have strategic capabilities, se

...
Book Review – Too Like the Lightning
by Jacob Falkovich — 2017-04-21T18:49:46.000Z

Readers who know me only through Putanumonit may suspect that my reading material consists of nothing but research papers in economics and post-rationalist blogs. That’s not true! I have a shelf full of books at home, without which I would be utterly unfuckable.

The books are actually my fiancee’s, almost none of them are mine. I have a Kindle and a New York Library card. But still: I read books!

My reading list is split 50-50 between non-fiction and fiction. Among the latter, my favorite genres are Irish fiction and speculative fiction, aka sci-fi. In the next couple of posts I’ll give my thoughts on some thought-provoking science fiction I had the pleasure of reading lately. We start today with Ada Palmer’s philosophical history thesis novel Too Like the Lightning which is the first in a series of at least four books .

This is not a critical review in the common sense of the word. I’m not going to score the books on a 1-5 scale or pretend that I know anything about comparative literary criticism. I like books that make me think, and I will write about that part first and foremost. I am writing both for those who read the book and for those who may want to in the future. I will keep spoilage to a minimum, referencing only plot points that show up in the first couple of chapters or that are immaterial to the main narrative.

A few words about the genre of science fiction. I love sci-fi, but some people see it as inferior to “serious” or “literary” fiction. The worst of these arguments are made by people who saw the Ender’s Game movie but never read the book. The best arguments may touch on an actual truth, one that is grounded in statistics.

A great work of literature can have a great plot and/or great insight into emotions and the human condition. A great work of science fiction can do both, but also a lot of other things. Some of my favorite sci-fi writers excel at building worlds (Iain M. Banks), some get me excited to learn a science (Greg Egan) or dive into history (Connie Willis). The best sci-fi, especially shorter format, has at its core an exciting speculative idea: what would the world look like if… humans didn’t need to sleep? if we invented a telepathy drug? if we decided that there’s nothing wrong with incest?

If literary fiction teaches you about humans through their (imagin...