trentmkelly/LessWrong-43k huggingface dataset:
New LW Meetups: Reading, Stockholm, and Suzhou
by FrankAdamek — 2015-10-09T16:11:27.185Z

This summary was posted to LW Main on October 2nd. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

First meetup in Lund: 19 October 2015 06:00PM
First meetup in Stockholm: 15 October 2015 03:00PM
Suzhou Meet-up: 28 October 2015 07:35PM
Reading First Meetup!: 07 October 2015 06:00PM

Irregularly scheduled Less Wrong meetups are taking place in:

Australia Online Hangout: 07 October 2015 06:00PM
Hamburg: 02 October 2015 07:00PM
Hamburg 2015 Q4: 16 October 2015 07:00PM
San Antonio Meetup - Discussion: 11 October 2015 12:30PM
Scotland October Meetup: 11 October 2015 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX - Caffe Medici - Board Games: 03 October 2015 01:30PM
Cambridge Less Wrong Meetup - Book Recommendations: 04 October 2015 03:30PM
Sydney Rationality Dojo - October: 04 October 2015 03:00PM
Tel Aviv: Hardware Verification and FAI: 28 October 2015 12:59AM
Tel Aviv: Black Holes after Jacob Bekenstein: 24 November 2015 08:00AM
Vienna: 17 October 2015 03:00PM
Vienna: 21 November 2015 04:00PM
[Vienna] Five Worlds Collide - Vienna: 04 December 2015 08:00PM
Washington, D.C.: Availability Heuristic: 04 October 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, Denver, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having yo...
Chatbots or set answers, not WBEs
by Stuart_Armstrong — 2015-10-09T15:48:36.000Z

A putative new idea for AI control; index here.

In a previous post, I talked about using a WBE to define a safe output for a reduced impact AI.

I've realised that the WBE isn't needed. Its only role was to ensure that the AI's output could have been credibly produced by something other than the AI - "I'm sorry, Dave. I'm afraid I can't do that." is unlikely to be the output of a random letter generator.

But a whole WBE is not needed. If the output is short, a chatbot with access to a huge corpus of human responses could function well. We can specialise it in the direction we need - if we are asking for financial advice, we can mandate a specialised vocabulary or train it on financial news sources.

So instead of training the reduced impact AI to behave as the 'best human advisor', we are are training it to behave as the 'luckiest chatbot'. This allows to calculate odds with greater precision, and has the advantage of no needing to wait for a WBE.

For some questions, we can do even better. Suppose we have a thousand different stocks, and are asking which one would increase in value the most during the coming year. The 'chatbot' here is simply an algorithm that picks a stock at random. So we now have an exact base rate - 1/1000 - and predetermined answers from the AI.

[EDIT:] Another alternative is to get online users to submit answers to the question. Then the AI selects the best answer from the choices. And if the AI is not turned on, a random answer is selected.
Toy model for wire-heading [EDIT: removed for improvement]
by Stuart_Armstrong — 2015-10-09T15:45:28.103Z

EDIT: these ideas are too underdeveloped, I will remove them and present a more general idea after more analysis.

This is a (very) simple toy model of the wire-heading problem to illustrate how it might or might not happen. The great question is "where do we add the (super)intelligence?"

Let's assume a simple model for an expected utility maximising agent. There's the input assessor module A, which takes various inputs and computes the agent's "reward" or "utility". For a reward-based agent, A is typically outside of the agent; for a utility-maximiser, it's typically inside the agent, though the distinction need not be sharp. And there's the the decision module D, which assess the possible actions to take to maximise the output of A. If E is the general environment, we have D+A+E.

Now let's make the agent superintelligent. If we add superintelligence to module D, then D will wirehead by taking control of A (whether A is inside the agent or not) and controlling E to prevent interference. If we add superintelligence to module A, then it will attempt to compute rewards as effectively as possible, sacrificing D and E to achieve it's efficient calculations.

Therefore to prevent wireheading, we need to "add superintelligence" to (D+A), making sure that we aren't doing so to some sub-section of the algorithm - which might be hard if the "superintelligence" is obscure or black-box.
A toy model of the control problem
by Stuart_Armstrong — 2015-10-09T15:31:07.000Z

This is a copy of this post; the relevant images can be found there. They were too large to repost here directly.

Jaan Tallinn has suggested creating a toy model of the control problem, so that it can be analysed without loaded concepts like "autonomy", "consciousness", or "intentionality". Here a simple (too simple?) attempt:

A controls B. B manipulates A.

Let B be a robot agent that moves in a two dimensional world, as follows:

B can push the yellow blocks around. It was programmed to want to push blocks into the black "hole" in the bottom right. The hole can contain arbitrarily many blocks, and B gets a penalty of some tiny ε for every step is takes. The programmer wanted B to simply move one box into the hole (for a reward of 1 once the task is complete), and so programmed it to receive a reward of 1 (and no more) if the box was in the hole, after the whole task was completed and B was turned off.

Unfortunately, B was programmed to operate in a noisy environment, so B actually has a probability of 0.99 of receiving the reward, for a given block in the hole. This means that if it pushes one block into the hole, it gets the reward with 0.99 probability; but if it pushes two blocks into the hole, it gets the reward with 1-(1-0.99)2 = 0.9999 probability. If it pushes all six blocks into the hole, it gets a reward with probability 0.999999999999. Therefore this tiny error has motivated B to try and push all blocks into the hole.

However, the designers were aware of the possibility of B misbehaving or there being a bug, and created agent A, a simple agent watching the situation. A's aim is to get robot B to to push one block into the hole, and stop there. It can turn B off. Its input sensor is the camera in the bottom left. This camera can see across the bottom row of squares (marked in light green), into the hole. Its algorithm is to turn B off as soon as it sees a block in the hole.

The intended outcome is that B pushes a block into the hole, the camera sees this, and A turns B off:

(see main link for image)

If we see B as using a model to determine its actions, there are several pieces of the model it could have:

The block world, including the camera, and itself.

The fact that there is a button that can turn it off.

The fact that A can press that button.

The fact that A takes the camera as input.

A's algorithm.

Now, it can't really do anything without 1. And if

...
How the virtual AI controls itself
by Stuart_Armstrong — 2015-10-09T15:05:44.000Z

A putative new idea for AI control; index here.

This seems very much the appropriate forum for this idea.

In previous posts, I posited AIs caring only about virtual worlds - in fact, being defined as processes in virtual worlds, similarly to cousin_it's idea. How could this go? We would want the AI to reject offers of outside help - be they ways of modifying its virtual world, or ways of giving it extra resources.

Let V be a virtual world, over which a utility function u is defined. The world accepts a single input string O. Let P be a complete specification of an algorithm, including the virtual machine it is run on, the amount of memory it has access to, and so on.

Fix some threshold T for u (to avoid the the subtle weeds of maximising). Define the statement:

r(P,O,V,T): "P(V) returns O, and either E(u|O)>T or O=∅"

And the string valued program:

Q(V,P,T): "If you can find that there exists a non-empty O such that r(P,O,V,T), return O. Else return ∅."

Here "find" and "E" are where the magic-super-intelligence-stuff happens.

Now, it seems to me that Q(V,Q,T) is the program we are looking for. It is uninterested in offers to modify the virtual world, because E(u|O)>T is defined over the unmodified virtual world. We can set it up so that the first thing it proves is something like "If I (ie Q) prove E(u|O)>T, then r(Q,O,V,T)." If we offer it more computing resources, it can no longer make use of that assumption, because "I" will no longer be Q.

Does this seem like a possible way of phrasing the self-containing requirements? For the moment, this seems to make it reject small offers of extra resources, and be indifferent to large offers.
Philosophical schools are approaches not positions
by casebash — 2015-10-09T09:46:31.711Z

One of the great challenges of learning philosophy is trying to understand the difference between different schools of thought. Often it can be almost impossible to craft a definition that is specific enough to be understandable, whilst also being general enough to convey to breadth of that school of thought. I would suggest that this is a result of trying to define a school as taking a particular position in a debate, when they would be better defined as taking a particular approach to answering a question.
Take for example dualism and monism. Dualists believe that there exist two substances (typically the material substance and some kind of soul/consciousness), while monists believe that there only exists one. The question of whether this debate is defined precisely enough to actually be answerable immediately crops up. Few people would object to labelling the traditional Christian model of souls which went to an afterlife as being a Dualist model or a model of our universe with no conscious beings whatsoever as being monist. However, providing a good, general definition of what would count as two substances and what would count as one seems extraordinarily difficult. The question then arises of whether the dualism vs. monism debate is actually in a form that is answerable.

In contrast, if Dualism and Monism are thought of as approaches, then there can conceivably exist some situations Dualism is clearly better, some situations where Monism is clearly better and some situations where it is debatable. Rather than labelling the situation to be unanswerable, it would be better to call it possibly unanswerable.

Once it is accepted that dualism and monism are approaches, rather than positions the debate becomes much clearer. We can define these approaches as follows: Monism argues for describing reality as containing a single substance, while dualism argues for describing reality as containing two substances: typically one being physical and the other being mental or spiritual. I originally wrote this sentence using the word ‘modelling’ instead of ‘describing’, but I changed it because I wanted to be neutral on the issue on whether we can talk about what actually exists or can only talk about models of reality. If it was meaningful to talk about whether one or two substances actually existed (as opposed to simply being useful models), the...
Emotional tools for the beginner rationalist
by Gleb_Tsipursky — 2015-10-09T05:01:57.952Z

Something that I haven't seen really discussed is what kind of emotional tools would be good for beginner rationalists. I'm especially interested in this topic since as part of my broader project of spreading rationality to a wide audience and thus raising the sanity waterline, I come across a lot of people who are interested in becoming more rational, but have difficulty facing the challenges of the Valley of Bad Rationality. In other words, they have trouble acknowledging their own biases and faults, facing the illusions within their moral systems and values, letting go of cached patterns, updating their beliefs, etc. Many thus abandon their aspiration toward rationality before they get very far. I think this is a systematic failure mode of many beginner aspiring rationalists, and so I wanted to start a discussion about what we can do about it as a community.

Note that this emotional danger does not feel intuitive to me or likely to many of you. In a Facebook discussion with Viliam Bur, he pointed out how he did not experience the Valley. I personally did not experience it that much either. However, based on the evidence of the Intentional Insights outreach efforts, this is a typical mind fallacy particular to many but far from all aspiring rationalists. So we should make an effort to address it in order to raise the sanity waterline effectively.

I'll start by sharing what I found effective in my own outreach efforts. First, I found it helpful to frame the aspiration toward rationality not as a search for a perfect and unreachable ideal, but as a way of constant improvement from the baseline where all humans are to something better. I highlight the benefits people get from this improved mode of thinking, to prime people to focus on their current self and detach themselves from their past selves. I highlight the value of self-empathy and self-forgiveness toward oneself for holding mistaken views, and encourage people to think of themselves as becoming more right, rather than less wrong :-)

Another thing that I found helpful was to provide new aspiring rationalists with a sense of community and social belonging. Joining a community of aspiring rationalists who are sensitive toward a newcomers' emotions, and help that newcomer deal with the challenges s/he experiences, is invaluable for overcoming the emotional strains of the Valley. Something ...
Fiction Considered Harmful
by abramdemski — 2015-10-08T18:34:05.857Z

Epistemic status: playing devil's advocate.

I wrote the following a couple of weeks back for a meet-up post, and Gunnar_Zarncke suggested I should turn it into a discussion post:

Fiction is not a lie, but it is a variety of untruth. It absorbs time and energy which could be spent on fact. Although we make a conscious distinction between fictional worlds and reality, we will often use fictional examples when evaluating real-life situations. It has been argued that we should learn to take joy in the world we actually live in. Why should we allow fiction to warp our view of reality?

Perhaps fiction offers a fun, relaxing break. I can understand this claim in two different ways. The first version is that reading fiction gives us a rest from serious thinking, restoring us in some way. So, is this really true? Often when we feel tired of thinking, we're really tired of thinking about some particular thing. We gain new mental energy when we switch to something else. We think this means we're unable to do productive work, and need to take a break; but often, we could continue to be productive on a sufficiently different task, which gave us the same variety as a "break" would. (This is anecdotal. I recall seeing a discussion of this in a lesswrong post, but didn't figure out which one.) Alternatively, if we really are exhausted, reading fiction might not be restoring our energy as much as taking a nap or perhaps meditating. In either case, the pro-fiction argument seems murky. Answering this question is difficult, because it's far from obvious why certain types of thinking seem to take "mental effort" and leave us feeling drained. (It seems it might be a mechanism for sensing high opportunity cost, or it might be due to depleting a physical resource in the brain.)

A second way to interpret this is that consuming fiction is closer to being an end, rather than a means. The joy which fiction creates, or the rich inner experience, may be a good in and of itself. Whether it's useful for restorative purposes or not, it's good that society keeps churning the fiction mill, because it's one of the things which makes lifeworthwhile. Some people will readily agree with this, while others will feel it's very close to advocating wireheading. At a recent LW meetup here in LA, one person argued that if you're goin...
Rationality Reading Group: Part K: Letting Go
by Gram_Stone — 2015-10-08T02:32:16.696Z

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part K: Letting Go (pp. 497-532). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

K. Letting Go

121. The Importance of Saying "Oops" - When your theory is proved wrong, just scream "OOPS!" and admit your mistake fully. Don't just admit local errors. Don't try to protect your pride by conceding the absolute minimal patch of ground. Making small concessions means that you will make only small improvements. It is far better to make big improvements quickly. This is a lesson of Bayescraft that Traditional Rationality fails to teach.

122. The Crackpot Offer - If you make a mistake, don't excuse it or pat yourself on the back for thinking originally; acknowledge you made a mistake and move on. If you become invested in your own mistakes, you'll stay stuck on bad ideas.

123. Just Lose Hope Already - Casey Serin owes banks 2.2 million dollars after lying on mortgage applications in order to simultaneously buy 8 different houses in different states. The sad part is that he hasn't given up - he hasn't declared bankruptcy, and has just attempted to purchase another house. While this behavior seems merely stupid, it brings to mind Merton and Scholes of Long-Term Capital Management, who made 40% profits for three years, and then lost it all when they overleveraged. Each profession has rules on how to be successful, which makes rationality seem unlikely to help greatly in life. Yet it seems that one of the greater skills is not being stupid, which rationality does help with.

124. The Proper Use of Doubt - Doubt is often regarded as virtuous for the wrong reason: because it is a sign of humility and recognition of your place in the hierarchy. But from a rationalist perspective, this is not why you should doubt. The doubt, rather, should exist to annihilate itself: to confirm the reason for doubting, or to show the doubt to be baseless. When you can no longer make progress in this respect, the doubt is no longer useful to you as a rationalist.

125. You Can Face...
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-10-07T16:49:15.549Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
11 October 2015 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

Cross-posted from the mailing list.

We'll be meeting to hang out, play games, converse, and any combination thereof.

If you would like to play a specific game, especially a long one or one with many players, feel free to send messages to the list to organize - they kick us out a little before 7 p.m.

Upcoming meetups:

Oct. 18: Utilitarianism

Oct. 25: Wise Sayings

Nov. 1: Fun & Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Group Rationality Diary, Oct. 6-18, 2015
by polymathwannabe — 2015-10-06T23:23:58.234Z

This is the public group rationality diary for October 6-18, 2015. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.
Crazy Ideas Thread - October 2015
by Gunnar_Zarncke — 2015-10-06T22:38:06.480Z

This thread is intended to provide a space for 'crazy' ideas. Ideas that spontaneously come to mind (and feel great), ideas you long wanted to tell but never found the place and time for and also for ideas you think should be obvious and simple - but nobody ever mentions them.

Rules for this thread:

Each crazy idea goes into its own top level comment and may be commented there.

Voting should be based primarily on how original the idea is.

Meta discussion of the thread should go to the top level comment intended for that purpose.

If you create such a thread do the following :

Use "Crazy Ideas Thread" in the title.

Copy the rules.

Add the tag "crazy_idea".

Create a top-level comment saying 'Discussion of this thread goes here; all other top-level comments should be ideas or similar'

Add a second top-level comment with an initial crazy idea to start participation.