trentmkelly/LessWrong-43k huggingface dataset:
Moral AI: Options
by Manfred — 2015-07-11T21:46:45.327Z

Epistemic status: One part quotes (informative, accurate), one part speculation (not so accurate).

One avenue towards AI safety is the construction of "moral AI" that is good at solving the problem of human preferences and values. Five FLI grants have recently been funded that pursue different lines of research on this problem.

The projects, in alphabetical order:

Vincent Conitzer:

Most contemporary AI systems base their decisions solely on consequences, whereas humans also consider other morally relevant factors, including rights (such as privacy), roles (such as in families), past actions (such as promises), motives and intentions, and so on. Our goal is to build these additional morally relevant features into an AI system. We will identify morally relevant features by reviewing theories in moral philosophy, conducting surveys in moral psychology, and using machine learning to locate factors that affect human moral judgments. We will use and extend game theory and social choice theory to determine how to make these features more precise, how to weigh conflicting features against each other, and how to build these features into an AI system. We hope that eventually this work will lead to highly advanced AI systems that are capable of making moral judgments and acting on them.

Techniques: Top-down design, game theory, moral philosophy

Owain Evans:

Previous work in economics and AI has developed mathematical models of preferences, along with algorithms for inferring preferences from observed actions. [Citation of inverse reinforcement learning] We would like to use such algorithms to enable AI systems to learn human preferences from observed actions. However, these algorithms typically assume that agents take actions that maximize expected utility given their preferences. This assumption of optimality is false for humans in real-world domains. Optimal sequential planning is intractable in complex environments and humans perform very rough approximations. Humans often don't know the causal structure of their environment (in contrast to MDP models). Humans are also subject to dynamic inconsistencies, as observed in procrastination, addiction and in impulsive behavior. Our project seeks to develop algorithms that learn human preferences from data despite the suboptimality of humans and the behavioral biases that influence human choice. We will test our algo

...
Rational vs Reasonable
by abramdemski — 2015-07-11T03:31:42.470Z

This post draws ideas from Personhood: A Game for Two or More Players on Melting Asphalt.

I've been lax in my attempt to write something for LW once weekly, but I hope to approximately continue nonetheless. I still have many posts planned -- the next one after this will likely be a rationality game that we've been playing at our weekly meetups in LA.

Last time, I talked about the distinction between associated and relevant. This time I'd like to talk about another distinction which comes up in rationality-conscious communication: that of rational vs reasonable.

Rationality has to do with figuring out what you actually want, being strategic about getting it, understanding what constitutes evidence, and so on. For more information, read the entire LessWrong archive.

Reasonableness is, in contrast, a social skill. It has to do with being able to give explanations for your actions, listening to and often accepting justifications for changing those actions, playing well on a team, behaving in a reliable and predictable manner, and dealing judiciously with guilt and responsibility.

I like reasonable people. Reasonableness is very valuable. It's probably a big part of what attracts me to rationalist circles in the first place: rationalists often value reasonableness more highly and are more careful to exercise it. Yet, rational and reasonable are two very different things. The most rational people are not the most reasonable people, or vice versa. I think it's worth examining in some detail how these two tails come apart.

Perhaps the largest difference comes from the way our explanations for behavior differ from the actual causes. Consciously or unconsciously, we engage in lies and half-truths when it comes to giving reasons for our behavior. This tendency is difficult to overcome because our brain does not ask for permission before generating these justifications. The incentives don't push us to total dishonesty, but they don't push us to total honesty, either. Evolution and our everyday social feedback conspire to make us give reasonable-sounding, socially defensible reasons in place of explaining the causes for our actions to-the-best-of-our-knowledge.

Aspiring rationalists will want to do away with some or all of this. However, this comes at a cost. To be perceived as reasonable by others, you will need to produce justifications...
Weekly LW Meetups
by FrankAdamek — 2015-07-10T15:50:55.669Z

This summary was posted to LW Main on July 3rd. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Australian Less Wrong Mega Meetup #2: 17 July 2015 07:00PM
Australia-wide Mega-Camp!: 17 July 2015 07:00PM
Hamburg : 17 July 2015 06:30PM
Oslo Meetup at Bitraf.: 18 July 2015 04:00PM
San Antonio, TX: 12 July 2015 01:30PM
San Francisco Meetup: Short Talks: 06 July 2015 06:15PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

[Melbourne] July Rationality Dojo: Bayesian Reasoning: 05 July 2015 03:30PM
[Seattle] Rationality Reading Group (57-61): 07 July 2015 06:30PM
Sydney Rationality Dojo - July: 05 July 2015 04:00PM
Sydney Meetup - July: 22 July 2015 06:30PM
Sydney Meetup - August: 26 August 2015 06:30PM
Sydney Meetup - September: 23 September 2015 06:30PM
Tel Aviv Meetup: Social & Board Games: 07 July 2015 07:00PM
Vienna: 18 July 2015 03:00PM
Washington, D.C.: American History: 04 July 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong ...
A Map: AGI Failures Modes and Levels 
by turchin — 2015-07-10T13:48:29.794Z

This map shows that AI failure resulting in human extinction could happen on different levels of AI development, namely, before it starts self-improvement (which is unlikely but we still can envision several failure modes), during its take off, when it uses different instruments to break out from its initial confinement, and after its successful take over the world, when it starts to implement its goal system which could be plainly unfriendly or its friendliness may be flawed.

AI also can halts on late stages of its development because of either technical problems or "philosophical" one.

I am sure that the map of AI failure levels is needed for the creation of Friendly AI theory as we should be aware of various risks. Most of ideas in the map came from "Artificial Intelligence as a Positive and Negative Factor in Global Risk" by Yudkowsky, from chapter 8 of "Superintelligence" by Bostrom, from Ben Goertzel blog and from hitthelimit blog, and some are mine.

I will now elaborate three ideas from the map which may need additional clarification.

The problem of the chicken or the egg

The question is what will happen first: AI begins to self-improve, or the AI got a malicious goal system. It is logical to assume that the goal system change will occur first, and this gives us a chance to protect ourselves from the risks of AI, because there will be a short period of time when AI already has bad goals, but has not developed enough to be able to hide them from us effectively. This line of reasoning comes from Ben Goertzel.

Unfortunately many goals are benign on a small scale, but became dangerous as the scale grows. 1000 paperclips are good, one trillion are useless, and 10 to the power of 30 paperclips are an existential risk.

AI halting problem

Another interesting part of the map are the philosophical problems that must face any AI. Here I was inspired after this reading Russian-language blog hitthelimit

One of his ideas is that the Fermi paradox may be explained by the fact that any sufficiently complex AI halts. (I do not agree that it completely explains the Great Silence.)

After some simplification, with which he is unlikely to agree, the idea is that as AI self-improves its ability to optimize grows rapidly, and as a result, it can solve problems of any complexity in a finite time. In particular, it will execute any goal system in a finite ti...
The Consequences of Dust Theory.
by Fivehundred — 2015-07-09T15:53:53.157Z

Dust theory implies that everything outside of my perception is in flux. Your experiences have to find themselves in a world in which they could have conceivably formed. Of course, you exist in every possible world which would produce that mindstate, but some are 'vaster' than others, leading you down the most probable courses.

Suppose that going to sleep or losing grasp of your surroundings opens a wider space of worlds you could exist in, which jumps you into another reality along with consistent memories of it. I can't figure out if this would be the case, or if my consciousness would most likely just dissolve, with only those beating trillion-to-one odds waking up in the morning. Or maybe my pool of 'experience' stays active when I sleep, even if I'm not aware of it. Either way (though I think Dust Theory is probably false) I'm afraid to go to sleep anymore.

I also do not understand the argument being made here: http://lesswrong.com/lw/1jm/getting_over_dust_theory/. Can someone explain to me please?

I posted these questions on other threads but I didn't get many answers. Sorry.

EDIT: Look, the first question boils down to: does my unconscious mind affect my measure? If so, than it isn't much different from being awake. If not, then all my problems seem to apply.

It occurs to me that not only would signing up for cryonics and then killing yourself before you could sleep is rational under these circumstances, but that the death of the universe can be escaped by simply rearranging your mind to believe it is in a universe where eternal life is possible, then ceasing its activity.
Beware the Nihilistic Failure Mode
by Gram_Stone — 2015-07-09T15:31:53.016Z

I have noticed that the term 'nihilism' has quite a few different connotations. I do not know that it is a coincidence. Reputedly, the most popular connotation, and in my opinion, the least well-defined, is existential nihilism, 'the philosophical theory that life has no intrinsic meaning or value.' I think that most LessWrong users would agree that there is no intrinsic meaning or value, but also that they would argue that there is a contingent meaning or value, and that the absence of such intrinsic meaning or value is no justification to be a generally insufferable person.

There is also the slightly similar but perhaps more well-defined moral nihilism; epistemological nihilism; and the not-unrelated fatalism.

Here, it goes without saying that each of these positions is wrong.

I recognize a pattern here. It seems that in each case the person who arrives at each of these positions has, in some informal sense, given up.

The idea finally came to my explicit attention after reading a passage in Nick Bostrom's Technological Revolutions: Ethics and Policy in the Dark. Bostrom writes:

If we want to make sense of the claim that physics is better at predicting than social science is, we have to work harder to explicate what it might mean. One possible way of explicating the claim is that when one says that physics is better at predicting than social science one might mean that experts in physics have a greater advantage over non‐experts in predicting interesting things in the domain of physics than experts in social science have over non‐experts in predicting interesting things in the domain of social science. This is still very imprecise since it relies on an undefined concept of “interesting things”. Yet the explication does at least draw attention to one aspect of the idea of predictability that is relevant in the context of public policy, namely the extent to which research and expertise can improve our ability to predict. The usefulness of ELSI‐funded activities might depend not on the absolute obtainable degree of predictability of technological innovation and social outcomes but on how much improvement in predictive ability these activities will produce. Let us hence set aside the following unhelpful question:

"Is the future of science or technological innovation predictable?"

A better question would be,

"How predictable are variou

...
Meetup : Warsaw integration: Barbecue on the beach
by mkf — 2015-07-09T14:19:29.630Z

Discussion article for the meetup : Warsaw integration: Barbecue on the beach

WHEN:
31 July 2015 06:30:00PM (+0200)

WHERE:
Warsaw

A casual meetup - integration on the beach on eastern side of Vistula river, near the Kryniczna bus stop. A map: https://goo.gl/maps/ZQ7Fa

Facebook event: https://www.facebook.com/events/1626726597575564/

Discussion article for the meetup : Warsaw integration: Barbecue on the beach
Should We Shred Whole-Brain Emulation?
by SteveG — 2015-07-09T10:02:23.620Z

I am opening this thread to test the hypothesis that SuperIntelligence is plausible but that Whole-Brain Emulations would most likely become obsolete before they were even possible.

Further, given the ability to do so, entities which were near to being Whole-Brain Emulations would rapidly choose to cease to be near Whole-Brain Emulations and move on to become something else.

I'll let people fire back with discussion and references before presenting more evidence. My hope is to turn this thread into something publishable in the end.
Meetup : Rationality Reading Group (62-66)
by SeekingEternity — 2015-07-09T05:45:00.825Z

Discussion article for the meetup : Rationality Reading Group (62-66)

WHEN:
13 July 2015 06:30:00PM (-0700)

WHERE:
Paul G. Allen Center (185 Stevens Way, Seattle, WA) Room 503

Reading group for Yudkowsky's "Rationality: AI to Zombies", which is basically an organized and updated version of the Sequences from LW (see http://wiki.lesswrong.com/wiki/Sequences).

The group meets to discuss the topics in the book, how to apply and benefit from them, and related topics in areas like cognitive biases, applied rationality, and effective altruism. You can get a copy of the book here: https://intelligence.org/rationality-ai-zombies/

The reading list for this week is five topics from the "Politics and Rationality" section of "How To Actually Change Your Mind". They are (actually 62-66, LW's auto-formatting is screwing it up):

Reversed Stupidity Is Not Intelligence

Argument Screens Off Authority

Hug the Query

Rationality and the English Language

Human Evil and Muddled Thinking

We previously covered the "Map and territory" sequence a few months ago, but please don't feel a need to have read everything up to this point to participate in the group.

Event is also on Facebook: https://www.facebook.com/events/1641756126111169/

We're meeting on the 5th floor. If you show up and the door into the room is locked, knock and look around for us elsewhere on the fifth floor if nobody answers. If the doors to the building are locked, try the other ones and don't believe the little red lights; try anyway. If the doors are, in fact, locked, we'll try to have somebody to let people in.

There's usually snacks at the meetup, though feel free to bring something. We usually get dinner afterward, around 9PM or so.

Discussion article for the meetup : Rationality Reading Group (62-66)
Meetup : Washington, D.C.: Fermi Estimates
by RobinZ — 2015-07-08T18:15:10.118Z

Discussion article for the meetup : Washington, D.C.: Fermi Estimates

WHEN:
12 July 2015 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

We'll be meeting to practice making Fermi estimates. We will congregate in the courtyard between 3:00 and 3:30 to start at 3:30. Please come with things in mind for others to estimate, ie, things you know the answer to or can find the answer to. The link includes examples.

As usual, if you're not interested in the meetup topic, come anyway to start side conversations.

Upcoming meetups:

Jul 19: Fun & Games

Jul 26: Meta Meetup

Aug 2: Optical Illusions

Discussion article for the meetup : Washington, D.C.: Fermi Estimates
Meetup : San Francisco Meetup: Board Games
by rocurley — 2015-07-08T17:15:02.954Z

Discussion article for the meetup : San Francisco Meetup: Board Games

WHEN:
13 July 2015 06:15:00PM (-0400)

WHERE:
1390 Market St. Apt 2115, San Francisco, CA

(I'm on the east coast right now and the website is confused: The meetup is at 18:15 PDT)

We'll be meeting to hang out and play board games at my apartment (our normal place doesn't reliably have tables). Please bring your own games: I have Dominion, Ticket to Ride, and Avalon.
You can call 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Board Games
Effective Altruism from XYZ perspective
by  — 2015-07-08T04:34:31.475Z

In this thread, I would like to invite people to summarize their attitude to Effective Altruism and to summarise their justification for their attitude while identifying the framework or perspective their using.

Initially I prepared an article for a discussion post (that got rather long) and I realised it was from a starkly utilitarian value system with capitalistic economic assumptions. I'm interested in exploring the possibility that I'm unjustly mindkilling EA.

I've posted my write-up as a comment to this thread so it doesn't get more air time than anyone else's summarise and they can be benefit equally from the contrasting views.

I encourage anyone who participates to write up their summary and identify their perspective BEFORE they read the others, so that the contrast can be most plain.
The Person As Input
by Eneasz — 2015-07-08T00:40:20.627Z

I. Humans are emotion-feeling machines.

I don’t mean that humans are machines that happen to feel emotions. I mean that humans are machines whose output is the feeling of emotions—“emotion-feeling” is the thing of value that we produce.

Not just “being happy." Then wireheading is the ultimate good, rather than the go-to utopia-horror example. But emotions must be involved, because everything else one can do are no more than a means to an end. Producing things, propagating life, even thinking. They all seem like endeavors that are useful, but a life of maximizing those things would suck. And the implication is that if we can create a machine that can do those things better than we can, it would be good to replace ourselves with that machine and set it to reproduce itself infinitely.

I recently saw a statement to the effect of “Art exists to produce feelings in us that we want, but do not get enough of in the course of normal life.” That’s what makes art valuable – supplementing emotional malnutrition. Such a thing exists because “to feel emotions” is the core function of humanity, and not fulfilling that function hurts like not eating does.

This is why (for many people) the optimal level of psychosis is non-zero. This is why intelligence is important – a greater level of intelligence allows a species to experience far more complex and nuanced emotional states. And the ability to experience more varieties of emotions is why it’s better to become more complex rather than simply dialing up happiness. It’s why disorders that prevent us from experiencing certain emotions are so awful (with the worst obviously being the ones that prevent us from feeling the “best” desires)

It’s why we like funny things, and tragic things, and scary things. Who wants to feel the way they feel after watching all of Evangelion?? Turns out – everyone, at some point, for at least a little bit of time!

It is why all human life has value. You do not matter based on what you can produce, or how smart you are, or how useful you are to others. You matter because you are a human who feels things.

My utility function is to feel a certain elastic web of emotions, and it varies from other utility functions by which emotions are desired in which amounts. My personality determines what ...
Crazy Ideas Thread
by Gunnar_Zarncke — 2015-07-07T21:40:48.931Z

This thread is intended to provide a space for 'crazy' ideas. Ideas that spontaneously come to mind (and feel great), ideas you long wanted to tell but never found the place and time for and also for ideas you think should be obvious and simple - but nobody ever mentions them.

This thread itself is such an idea. Or rather the tangent of such an idea which I post below as a seed for this thread.

Rules for this thread:

Each crazy idea goes into its own top level comment and may be commented there.

Voting should be based primarily on how original the idea is.

Meta discussion of the thread should go to the top level comment intended for that purpose.

If this should become a regular thread I suggest the following :

Use "Crazy Ideas Thread" in the title.

Copy the rules.

Add the tag "crazy_idea".

Create a top-level comment saying 'Discussion of this thread goes here; all other top-level comments should be ideas or similar'

Add a second top-level comment with an initial crazy idea to start participation.