trentmkelly/LessWrong-43k huggingface dataset:
Trapping AIs via utility indifference
by Stuart_Armstrong — 2012-02-28T19:27:25.607Z

If you wanted to check whether your creations were prone to misbehaving, one way would be to leave a big juicy apple hanging tantalisingly from a very visible tree, turn your back for a while, and see what they get up to.

That would work if you were God and your creations were essentially children. But if the creator is us and the creation is a super-intelligent AI, such 'honey-pots' and unlikely to work. They are tricks, and we shouldn't rely on being able to trick a being more intelligent than ourselves - unless its intelligence doesn't matter.

We can use utility indifference to construct such traps. Utility indifference can be informally summarised as:

"There is a stochastic event X which can output 0 or 1. We wish the AI act as if it believed that X=0, with certainty. It's problematic for the AI to have incorrect beliefs, so the result is achieved through the definition of the AI's utility function."

Consider the following situation: a master AI that will create the source code of another, autonomous AI. The master AI is setup to produce the code of a Friendly AI, or a reduced impact AI - but we're not sure if the master AI is trustworthy. It would be good to go through the code very carefully before unleashing it (or erasing it). But if the master AI knows this (and we must assume it does), then it will either obfuscate the dangerous parts or put traps in it, making it risky for us to look too closely.

Then implement the following setup. The code travels through some stochastic event X, such that if X=0, the code is immediately transferred into a disciple AI, completely autonomous, which is at liberty to immediately modify its own code and erase any trace of what it was. If X=1, the code is instead copied to our hard drives for analysis. The AI is given some reason to keep the code short - maybe there is an increasing chance of the code getting erased the longer it is.

On top of this, we implement utility indifference: the master AI will act as if it believed X=0 with certainty, while we have setup X so that X=1 with near-certainty. Then the master AI will be motivated to produce the shortest code compatible with its utility function, in the 'knowledge' that this code will be copied into a powerful autonomous AI, and will never be seen by humans. It has no incentive to trap its code - quite the reverse, as it wants to keep it short.

A...
Rationality & Startups - The Workshop
by Alexandros — 2012-02-28T11:25:52.384Z

I have been given the opportunity to prepare a workshop for the General Assembly team in London. General Assembly is geared towards education of entrepreneurs and aspiring entrepreneurs and have been very successful in New York, now expanding to London. The workshops are 90 minutes long, and usually gather anywhere from 15 to 35 people who have paid to attend.

While I considered doing something on concrete coding skills, I think by far the superior alternative (for myself and the audience) is to do a crash course on cognitive bias as it relates to startups, maybe throw in some other topics on rationality in a similar context. I am fairly confident that startups are an excellent testing ground for extreme rationality as they require exceptionally quick assimilation of new skills and knowledge, as well as demand rapid decisions with incomplete information.

So, as part of the brainstorming for this, here are my questions for you:

1.Do you think educating startup founders on cognitive bias/rationality will help them improve their outcomes?

2.Which biases would especially affect startups? Which of these can be mitigated (either by knowing about them or by utilising explicit strategies)?

3.What is a good way to use 90 minutes to get this information across?

4.What prior material exists to introduce rationality in a fast-paced manner? What prior material exists that relates startups to rationality?

5.Other relevant thoughts welcome

Should I go ahead with this, I will of course make the deck available for any others who may want to do similar presentations elsewhere.
My summary of Eliezer's position on free will
by Solvent — 2012-02-28T05:53:10.432Z

I'm participating in a university course on free will. On the online forum, someone asked me to summarise Eliezer's solution to the free will problem, and I did it like this. Is it accurate in this form? How should I change it?

“I'll try to summarise Yudkowsky's argument.

As Anneke pointed out, it's kinda difficult to decide what the concept of free will means. How would particles or humans behave differently if they had free will compared to if they didn't? It doesn't seem like our argument is about what we actually expect to see happening.

This is similar to arguing about whether a tree falling in a deserted forest makes any noise. If two people are arguing about this, they probably agree that if we put a microphone in the forest, it would pick up vibrations. And they also agree that no-one is having the sense experience of hearing the tree fall. So they're arguing over what 'sound' means. Yudkowsky proposes a psychological reason why people may have that particular confusion, based on how human brains work.

So with respect to free will, we can instead ask the question, “Why would humans feel like they have free will?” If we can answer this well enough, then hopefully we can dissolve the original question.

It feels like I choose between some of my possible futures. I can imagine waking up tomorrow and going to my Engineering lecture, or staying in my room and using Facebook. Both of those imaginings feel equally 'possible'.

Humans execute a decision making algorithm which is fairly similar to the following one.

List all your possible actions. For my lecture example, that was “Go to lecture” and “Stay home.”

Predict the state of the universe after pretending that you will take each possible action. We end up with “Buck has learnt stuff but not Facebooked” and “Buck has not learnt stuff but has Facebooked.”

Decide which is your favourite outcome. In this case, I'd rather have learnt stuff. So that's option 2.

Execute the action associated with the best outcome. In this case, I'd go to my lecture.

Note that the above algorithm can be made more complex and powerful, for example by incorporating probability and quantifying your preferences as a utility function.

As humans, our brains need the capacity to pretend that we could choose different things, so that we can imagine the outcomes, ...
Troubles With CEV Part2 - CEV Sequence
by diegocaleiro — 2012-02-28T04:19:55.606Z

The CEV Sequence Summary: The CEV sequence consists of three posts tackling important aspects of CEV. It covers conceptual, practical and computational problems of CEV's current form. On What Selves Are draws on analytic philosophy methods in order to clarify the concept of Self, which is necessary in order to understand whose volition is going to be extrapolated by a machine that implements the CEV procedure. Troubles with CEV part1 and Troubles with CEV part2 on the other hand describe several issues that will be faced by the CEV project if it is actually going to be implemented. Those issues are not of conceptual nature. Many of the objections shown come from scattered discussions found on the web. Finally, six alternatives to CEV are considered.

Troubles with CEV Summary: Starting with a summary of CEV, we proceed to show several objections to CEV. First, specific objections to the use of Coherence, Extrapolation, and Volition. Here Part1 ends. Then, in Part2, we continue with objections related to the end product of performing a CEV, and finally, problems relating to the implementation of CEV. We then go on with a praise of CEV, pointing out particular strengths of the idea. We end by showing six alternatives to CEV that have been proposed, and considering their vices and virtues.

Meta: I think Troubles With CEV Part1 and Part2 should be posted to Main. So on the comment section of Part2, I put a place to vote for or against this upgrade.

Troubles with CEV Part2

5) Problems with the end product

5a) Singleton Objection. Even if all goes well and a machine executes the coherent extrapolated volition of humanity, the self modifying code it is running is likely to become the most powerful agent on earth (including individuals, governments, industries and other machines) If such a superintelligence unfolds, whichever goals it has (our CE volitions) it will be very capable of implementing. This is a singleton scenario. A singleton is “[T]he term refers to a world order in which there is a single decision-making agency at the highest level. Among its powers would be (1) the ability to prevent any threats (internal or external) to its own existence and supremacy, and (2) the ability to exert effective control over major features of its domain (including taxation and territorial allocation).”. ...
Troubles With CEV Part1 - CEV Sequence
by diegocaleiro — 2012-02-28T04:15:30.309Z

The CEV Sequence Summary: The CEV sequence consists of three posts tackling important aspects of CEV. It covers conceptual, practical and computational problems of CEV's current form. On What Selves Are draws on analytic philosophy methods in order to clarify the concept of Self, which is necessary in order to understand whose volition is going to be extrapolated by a machine that implements the CEV procedure. Troubles with CEV part1 and Troubles with CEV part2 on the other hand describe several issues that will be faced by the CEV project if it is actually going to be implemented. Those issues are not of conceptual nature. Many of the objections shown come from scattered discussions found on the web. Finally, some alternatives to CEV are considered.

Troubles with CEV Summary: Starting with a summary of CEV, we proceed to show several objections to CEV. First, specific objections to the use of Coherence, Extrapolation, and Volition. Here Part1 ends. Then, in Part2, we continue with objections related to the end product of performing a CEV, and finally, problems relating to the implementation of CEV. We then go on with a praise of CEV, pointing out particular strengths of the idea. We end by showing six alternatives to CEV that have been proposed, and considering their vices and virtues.

Meta: I think Troubles With CEV Part1 and Part2 should be posted to Main. So on the comment section of Part2, I put a place to vote for or against this upgrade.

Troubles with CEV Part1

Summary of CEV

To begin with, let us remember the most important slices of Coherent Extrapolated Volition (CEV).

“Friendly AI requires:

1. Solving the technical problems required to maintain a well-specified abstract invariant in a self-modifying goal system. (Interestingly, this problem is relatively straightforward from a theoretical standpoint.)

2. Choosing something nice to do with the AI. This is about midway in theoretical hairiness between problems 1 and 3.

3. Designing a framework for an abstract invariant that doesn't automatically wipe out the human species. This is the hard part.

But right now the question is whether the human species can field a non-pathetic force in defense of six billion lives and futures.”
“Friendliness is the easiest part of the problem to explain - the part that says what we want. Like explaining wh

...
Brazilians, unite! and what is IERFH (portuguese)
by diegocaleiro — 2012-02-28T04:05:32.521Z

Hi anglophones, this topic is only for brazilians, so someone may post in portuguese and part of this is in portuguese (We will translate it to english if necessary when the time comes).

Hello Brazilians, I'm creating this topic because some misallocated questions were posed on this one.

First, the numbers.In Less Wrong:

Me, Gust, Paulovsk, zecaurubu, Gracunha, Mexamark, dyokomizo.

From IERFH (Instituto Ética, Racionalidade, e o Futuro da Humanidade)

Leo Arruda, João Fabiano, me, Pierre , Jonatas Muller, Pablo, Lauro (paralelo), Rafael, plus 3 others.

This makes us at least 17, almost one every 10 million people (not great...)

Some of us are in São Paulo. About 10.

Eventually, this topic may attract some others, and we can create a meeting.

Now, regarding the question a few of you did, IERFH's mission:

Gerar alto impacto positivo no longo prazo, produzindo conhecimentos e reunindo pessoas que contribuam para melhor pensar as questões éticas que irão definir o futuro da humanidade.

About: Somos um time comprometido com fazer o mundo melhor, agora e no futuro. Para isso, estamos reunindo a comunidade brasileira de racionalistas, utilitaristas, transhumanistas, e outros entusiastas e tranformando ideais e teorias em ações. Ao lado de grandes organizações internacionais de caridade, tecnologia e ética, nos propomos a ser o vetor dos esforcos brasileiros nesses campos. O IERFH opera em 3 frentes: o estudo do que é bom e deve ser buscado e preservado: a Ética Pura e Aplicada; as maneiras mais eficientes de raciocínio, tomada de decisões e os seus erros mais comuns: a Racionalidade Epistêmica e Prática; e por fim como aplicar estes campos para garantir a plena realização de todo o potencial humano: o Futuro da Humanidade

Racionalidade: Ser racional é conseguir conquistar, com pouco dispêndio de recursos, aquele que se deseja, entre todos os cenários possíveis que poderiam ter ocorrido. Racionalidade epistêmica é a capacidade de entender com o mundo atual; Racionalidade prática, a capacidade de guiar o mundo atual em direção ao mundo desejado. Para ser racional, duas capacidades são fundamentais, a capacidade de desviar dos bias cognitivos, falhas sistemá...
'Facing the Singularity' podcast
by lukeprog — 2012-02-28T03:22:45.708Z

My online mini-book Facing the Singularity now has a podcast. Ratings and reviews on iTunes will be much appreciated, so as to direct people toward a rationality-informed approach to intelligence explosion.

Jaan Tallinn has been passing my chapters around to people because they are concise explanations of key lemmas in the standard arguments on the need for Friendly AI. This is gratifying because it's exactly the purpose for which I'm writing them, and I encourage others to send people to these chapters as well.

(I'm currently writing the final two chapters of the online book and recording readings of the other chapters for the podcast. A volunteer is doing the audio editing.)
[SEQ RERUN] Qualitatively Confused
by MinibearRex — 2012-02-28T02:59:08.243Z

Today's post, Qualitatively Confused was originally published on 14 March 2008. A summary (taken from the LW wiki):

Using qualitative, binary reasoning may make it easier to confuse belief and reality; if we use probability distributions, the distinction is much clearer.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Quotation is not the Referent, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : West LA Meetup - Reasoning About Evolutionary Psychology & History
by  — 2012-02-28T02:20:33.265Z

Discussion article for the meetup : West LA Meetup - Reasoning About Evolutionary Psychology & History

WHEN:
29 February 2012 07:00:00PM (-0800)

WHERE:
10850 West Pico Blvd, Los Angeles, CA 90064

When: 7:00pm - 9:00pm Wednesday, February 29th.

Where: The Westside Tavern in the upstairs Wine Bar (all ages welcome), located inside the Westside Pavillion on the second floor, right by the movie theaters.

Parking is free for 3 hours.

Discussion Topic: This week, we will talk about how to (and not to) reason about historical events, with an emphasis on evolutionary psychology. Take a look at some of the posts tagged evpsych for some thought food.

This is also a good time to browse recent posts, which also make for good discussion!

Don't worry if you don't have time to read anything, or even if you've never read any Less Wrong! Bring a friend! The atmosphere is casual, and good, intelligent conversation with friendly people is guaranteed.

I will bring a whiteboard with Bayes' Theorem written on it.

Discussion article for the meetup : West LA Meetup - Reasoning About Evolutionary Psychology & History
Mike Darwin on the Less Wrong intelligentsia
by Synaptic — 2012-02-28T01:59:31.620Z

He has resumed posting at his blog Chronopause and he is essential reading for those interested in cryonics and, more generally, rational decision-making in an uncertain world.

In response to a comment by a LW user named Alexander, he writes:

I have no objection to you promoting Chronosphere on LessWrong and would strongly encourage you to do so. But my guess is that you will find it an unrewarding, and possibly punishing exercise.... I’m not being trite when I say that people do NOT like reality.... Life is scary and hard, in fact, it is absolutely terrifying if looked at objectively, and things like cryonics and religion ostensibly exist to REDUCE that terror and to make to life (and death) more bearable.

For that to work, there has to be a happy sense of optimism and trust that everything is going to work out just fine. This makes sense, and, arguably, is even necessary, because the alternative would be that everyone who chose cryonics would have to more than a customer – they would have be involved – at least to some extent – as an activist. That’s asking an awful lot. But most of all, it is asking that people who opt for cryonics give up the comfort of the fantasy that they can be customers, buy the product, pay their money every month or every quarter, and that they will then be taken care of when the need arises. It means that they will have to do two exceedingly difficult and uncomfortable things:

1) Confront and understand the reality of the the many shortcomings and uncertainties of contemporary cryonics and live with them.

2) Work industriously, creatively and continuously to overcome those shortcomings.

It is very, very hard to create and sustain an organization and a community where the above two dictums are practiced. It is not only hard for the community members, it is hard for the community leaders.

(Sidenote: This reminds me of what Luke considers his most difficult day-to-day tasks.)

...[I]f you expect to see cryonics organizations to have validated the quality of their brain cryopreservation technology under a wide variety of real world conditions, you will be disappointed. If you expect to see contingency planning for serious, inevitable near-term existential risks, such as pandemic disease, you will also be disappointed. Similarly, if you expect to see technologies in place to mitiga

...
Meetup : Fort Collins, Colorado Meetup Wedneday 7pm
by EvelynM — 2012-02-28T00:22:07.964Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm

WHEN:
29 February 2012 07:00:00PM (-0700)

WHERE:
144 North College Avenue, Fort Collins, CO 80524

What are your cool projects?

Come meet interesting people and up your game.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm
Meetup : Fort Collins, Colorado Meetup Wedneday 7pm
by EvelynM — 2012-02-28T00:21:57.298Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm

WHEN:
29 February 2012 07:00:00PM (-0700)

WHERE:
144 North College Avenue, Fort Collins, CO 80524

What are your cool projects?

Come meet interesting people and up your game.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm
Meetup : University of Chicago (Again)
by falenas108 — 2012-02-27T19:46:22.215Z

Discussion article for the meetup : University of Chicago (Again)

WHEN: 03 March 2012 01:00:00PM (-1300)

WHERE: 1135 E. 57th Street Chicago, IL 60637

People were interested in having meetups more frequently, so we're experimenting with having another one this week.

To get there, walk into the building labeled Reynolds Club/Hutchinson Commons, then turn into the first room on the right.

Discussion article for the meetup : University of Chicago (Again)
General Physics Discussion, Explainations and Help.
by  — 2012-02-27T17:01:40.795Z

Hello everyone.

I was wondering why it was that we don't have any discussion of general physics. In the spirit of general education here I am.

Feel free to post questions of any kind regarding physics or mathematics useful in physics, I know I will help best I can and I hope others will too. As for myself I have a question:

I have trouble visualizing electrostatic systems. Electrodynamics I find easy (I am well versed in vector field analysis and calculus), but I have completely missed creating a mental model for electrostatics. I know the equations but it feels like a teachers password. Anything will probably help.
Grad School?
by eggman — 2012-02-27T11:49:23.079Z

Previous article: Which college major?

Note: I don't have a trust fund. Even though I said I did in the last post. Apparently I misunderstood what that was, and I just have RESPs to cover my undergrad. Please take that into account.

Building on my last post, I've concluded to pursue either a major or a double major in one or two of the following areas for my undergraduate degree:

Economics;

Computer Science;

Cognitive Science/Psychology/Behavioural Neuroscience;

Mathematics/Statistcs;

Biotechonology;

Engineering. Engineering would be another 5 years for me and I don't know what the job prospects are like. Is it worth it? I may also possibly do a minor in philosphy/mathematics/physics.

Grad school appeals to me. Current considerations are in neuroeconomics or some other neuroscience, economics, or some hard science (still undecided). The only graduate degree in the humanities I would pursue would be in Philosophy or in Philosophy of Science. Professional programs that I might consider going into are medicine (least likely), M.B.A., Law, healthcare (e.g., physiotherapy/OT; nursing practitioner, etc.), teaching (preferred but unlikely).

Foreseeable problems with grad work:

1) It takes a long time to publish in peer-reviewed journals in academia Instead, if I really turn out to be some sort of prodigy, I will have a greater intellectual and ethical impact by just becoming a better rationalist and publishing well-written and well-researched blog posts, like SIAI, FHI, and Overcoming Bias do. And then making some fat cash on the side and donating it to the most effective causes. Which career for me would lead to this I don't know.

2) 80,000 hours pointed out in one blog post (I forget which one) that grad or professional schooling can be more time/effort/work than it's worth. This would be especially true if I'm saddled with debt and cannot find employment with high remuneration.

3) The potential jobs that I would pursue could be outsourced, or have humans being replaced by robots/computers. This would make all that extra school a waste of time, obviously.

4) Having a graduate degree somehow makes my resume look less attractive than just having an undergrad. I don't understand the rationale behind this phenomenon, but I've heard it exists in some fields.

Do you have any recommendations/advice? Specifically:

What is a waste of time? What is a great use of my time

...